-- Logs begin at Thu 2022-09-01 15:37:54 UTC, end at Thu 2022-09-01 17:02:44 UTC. -- Sep 01 16:13:12.269978 dv-cinder-836027-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 01 16:13:12.279270 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 1 16:13:12 2022] *** Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: nodename: dv-cinder-836027-9-iscsi Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: machine: x86_64 Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: clock source: unix Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: pcre jit disabled Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: detected number of CPU cores: 4 Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: current working directory: / Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: detected binary path: /usr/bin/uwsgi-core Sep 01 16:13:12.281265 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: your processes number limit is 47680 Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: your memory page size is 4096 bytes Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: detected max file descriptor number: 2048 Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: lock engine: pthread robust mutexes Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: thunder lock: enabled Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 01 16:13:12.281905 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 01 16:13:12.320612 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: Python main interpreter initialized at 0x558d01bec9c0 Sep 01 16:13:12.320612 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: python threads support enabled Sep 01 16:13:12.320612 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: your server socket listen backlog is limited to 100 connections Sep 01 16:13:12.320612 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: your mercy for graceful operations on workers is 90 seconds Sep 01 16:13:12.320939 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: mapped 403077 bytes (393 KB) for 2 cores Sep 01 16:13:12.321037 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: *** Operational MODE: preforking *** Sep 01 16:13:12.321307 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: *** uWSGI is running in multiple interpreter mode *** Sep 01 16:13:12.321307 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: spawned uWSGI master process (pid: 81607) Sep 01 16:13:12.321511 dv-cinder-836027-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 01 16:13:12.324703 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: spawned uWSGI worker 1 (pid: 81619, cores: 1) Sep 01 16:13:12.324703 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: spawned uWSGI worker 2 (pid: 81620, cores: 1) Sep 01 16:13:12.324703 dv-cinder-836027-9-iscsi devstack@keystone.service[81607]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 01 16:13:16.647789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:13:16.647789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Sep 01 16:13:16.648595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:13:16.648595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Sep 01 16:13:16.648724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:13:16.648724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: from cryptography.utils import int_from_bytes Sep 01 16:13:16.649174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 01 16:13:16.649174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: from cryptography.utils import int_from_bytes Sep 01 16:13:16.701244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:13:16.701427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:13:16.701622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] command line args: [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:13:16.701894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:13:16.701998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 01 16:13:16.702081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ================================================================================ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:13:16.702152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 01 16:13:16.702313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] admin_token = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.702398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] command line args: [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 01 16:13:16.702559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_dir = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.702637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 01 16:13:16.702748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.702834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ================================================================================ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 01 16:13:16.702932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_source = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] debug = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_dir = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_source = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.703913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] debug = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_config_append = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.704948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_dir = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_config_append = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_dir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.705955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.706951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.707973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.708819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] publish_errors = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.709954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] publish_errors = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] standard_threads = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.710931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.711083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.711175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.716943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] standard_threads = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_journal = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_json = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_stderr = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_syslog = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_journal = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_json = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.717921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_stderr = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.717921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_syslog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.718137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 01 16:13:16.718279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.718989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.719973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.720928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.721827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.722962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.723998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.724949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.725898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.726949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.727976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.728994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.729924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.730898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.731959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.732961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.733942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.734934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.735995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.736946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.737937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.external = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.external = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.738952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.password = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.token = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.739945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.password = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.token = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.740886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.741864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.742881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.743894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.744907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.745859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.746977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.747962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.748926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.749963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.750894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.751877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.752993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.753944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.754975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.755137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.755241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.755398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.755477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.756993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.757911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.758916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.759894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.760042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.760125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.760811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.760943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.761921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.762978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.763947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.764902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.765912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.766974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.767950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.768902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.769989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.770985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.771986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.772940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.773982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.774951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.775921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.776541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.776894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.777867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.778992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.779853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.780859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.781982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.782924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.783925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.784958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.785937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.786895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.787866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.788988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.789930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:13:16.790425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.790872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.791114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.791375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 01 16:13:16.791582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 01 16:13:16.820040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.820273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.820499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.820757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.820993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.821220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.829419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.830414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.857881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.858112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.858701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.858935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.915778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.915934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.965877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.966124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.966381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:16.966620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.125743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.125968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.126968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.127208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.157942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.158914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.169823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.169931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.183627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.183770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.183853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.183928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.184995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 01 16:13:17.202479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.202619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.202707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:13:17.202797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:13:17.203022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.203108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.203247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:13:17.203332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 01 16:13:17.203526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.203612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.203825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.204044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.204137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.204310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.204404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.204576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.214941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.215025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.221909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.222057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.224841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.224967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.225132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.225253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.225423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.225539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.225711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.225822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.226915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.227911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.251074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.252037 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.252574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.252865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.253606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.253838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.257530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.257821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.258113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.258385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.258671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.258957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.265130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.265425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.266292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.266721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.269525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.269821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.270108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.270431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.270641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.270725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.270937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271532 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.271806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.273883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.274175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.274465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.288843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.289129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.290353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.290637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.293043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.294793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.296307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.296610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.297884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.298183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.300878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.302511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.304013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.305607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.308205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.308518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.308809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.309108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.313400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.326108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.326402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.326672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.326936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.328070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.328358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.328624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.328895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.333830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.335466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.335645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.337003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.337286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.337378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.339087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.339388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.340016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.340313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.340710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.340997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.341293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.342086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.342373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.342663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.342942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.343255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.352514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.352803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.353187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.353474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.353759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.354229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.354529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.354973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.355284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.355568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.355850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.356136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.356415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.356716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.370899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.373112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.373879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.376256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.376785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.378228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.379174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.379883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.380169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.380455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.380631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.382120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.382406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.382689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.387893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.389374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.390127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.391633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.392515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.392809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.393099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.393383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.394572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.394874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.395170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.395461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.401326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.401620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.401903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.403868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.404162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.404443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.408854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.409144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.409430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.409711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.409995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.410283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.410571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.410849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.411131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.411359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 01 16:13:17.411445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.411646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.411730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.411928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.412754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.413029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.413305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.413584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.413859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.414130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.414391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.414654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.414932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 01 16:13:17.448023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558d01bec9c0 pid: 81620 (default app) Sep 01 16:13:17.451285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df282e-b1ac-4af7-a0ef-5afc771bbb65 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:17.451383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558d01bec9c0 pid: 81619 (default app) Sep 01 16:13:17.451574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df282e-b1ac-4af7-a0ef-5afc771bbb65 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:17.451845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df282e-b1ac-4af7-a0ef-5afc771bbb65 None None] PATH_INFO: `/v3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:17.456000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Sep 1 16:13:17 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:25.030252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:25.030773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:25.030908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:25.035174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:13:25.041149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:25.041683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:25.041683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:25.113414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: INFO dbcounter [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] Registered counter for database keystone Sep 01 16:13:25.134991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG oslo_db.sqlalchemy.engines [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81620) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:13:25.141064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writer thread running {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:13:25.324853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81620) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:13:25.325154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81620) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:13:25.382160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:25.384649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:25.513031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 01 16:13:25.994184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:26.064023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: INFO dbcounter [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] Registered counter for database keystone Sep 01 16:13:26.082744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG oslo_db.sqlalchemy.engines [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81619) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 01 16:13:26.089502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writer thread running {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 01 16:13:26.209488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83e6ab-ddf9-4714-9372-53f076fd5815 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:26.214544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:26.215205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:26.215486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:26.215758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:26.223762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:26.276739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:13:26.277087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3d0e4-c740-46a5-b83d-5dc5966e2c46 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:26.286955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 1 16:13:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:13:26.293638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:26.308190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e77ca1-25ec-44d8-913c-cfcaaa2df2ed None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:26.312622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:26.313704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:26.313981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:26.314257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] PATH_INFO: `/v3/projects/18a3d39073f24e449904bb491fbe8869` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:26.315822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] RBAC: Authorizing `identity:get_project(project_id=18a3d39073f24e449904bb491fbe8869)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:26.369712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 01 16:13:26.370058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfc59a-e7d6-49f3-b86f-e065b0fc4a66 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:26.374515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Sep 1 16:13:26 2022] GET /identity/v3/projects/18a3d39073f24e449904bb491fbe8869 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:28.265979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b6d9-8f5e-42b8-b8ab-39d2ab8d6271 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:28.266557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b6d9-8f5e-42b8-b8ab-39d2ab8d6271 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:28.266557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528b6d9-8f5e-42b8-b8ab-39d2ab8d6271 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:28.270656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:28.276992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:28.277415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:28.277536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:28.297389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:28.298922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:28.307780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 01 16:13:28.585106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528b6d9-8f5e-42b8-b8ab-39d2ab8d6271 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:28.597281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528b6d9-8f5e-42b8-b8ab-39d2ab8d6271 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:28.601680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:28.602844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:28.602844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:28.602844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:28.610382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:28.612198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc18d1a8-d399-4194-bafe-4d07ca45edbc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:28.621376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Sep 1 16:13:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:13:28.626760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:28.636652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e00b-f84a-4c22-b109-072054baf26c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:28.641004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:28.641626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:28.641904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:28.642175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:28.643504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:28.645328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f60e3-3f9a-4688-8fe5-f574692ccc53 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:28.646836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Sep 1 16:13:28 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:30.626056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be93bd-4cef-46ef-b59b-65ec61615ef9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:30.626608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be93bd-4cef-46ef-b59b-65ec61615ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:30.626608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be93bd-4cef-46ef-b59b-65ec61615ef9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:30.627931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:30.633679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:30.634076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:30.634301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:30.645780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f383-9835-4a18-bb71-4e755265220a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:30.646054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f383-9835-4a18-bb71-4e755265220a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:30.646326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63f383-9835-4a18-bb71-4e755265220a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:30.647675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:13:30.652622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:30.652760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:30.652927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:30.653203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:30.653864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:30.696022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG passlib.handlers.bcrypt [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81619) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 01 16:13:30.696405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG passlib.handlers.bcrypt [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81619) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 01 16:13:30.732930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:30.745085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:30.746649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:30.845944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:31.002723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.026626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.107219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54441fa8-3e12-4235-b8ff-d778264274a2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.109432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.141267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.141908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.142186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.142460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.153544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:31.162431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:31.163931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:31.164771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:31.169940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.210437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d73aea7-d03d-4434-b261-b5b5a7dbbb3f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.212744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.252632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.253393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.253649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.254393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.254695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.271858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc08ab7-ae4e-4351-a23b-623ed407279e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.307422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.308033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.308311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.308584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.309248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:31.309549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.310935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.322528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:31.323263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.324866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.328002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.423859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:13:31.426888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f88fe4-ee03-47b2-a95f-6e36c46d76db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.430042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.432809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.463981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.464623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.464902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.465176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.466315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.467828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.469412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:31.474853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.488892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.528881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8f40f-67dc-4f71-b46e-b90375e4dbfe None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.562427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.563312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.563594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.563863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.573551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa45ce-2d93-4d93-ba53-2bc7f7c11a1e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.581945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.583480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:13:31.603222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:31.604101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:13:31.604918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:13:31.604918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:13:31.604918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:31.604918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:13:31.607871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.608507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.608787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.609183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.610003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.610195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.611689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.613394 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:31.619640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:31.668370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.708312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:31.714849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29fbb9-5585-4903-a5dc-78cafad3f032 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.727654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e6200-cf37-4986-99b9-af7f6fcb9aab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:31.748944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.749545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.749822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.750095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.750734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:31.751033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.752434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cdf48-3d53-4582-abd9-d4677492a056 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.761295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:31.761900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:31.762178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:31.762507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:31.763271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.domain.id=default, user.id=f9fb5099a6c24abebce19b1a9dfe02b2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:31.763570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:31.764984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d6713-61b6-4783-8ba6-e1d4bdd1aa03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:31.769916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:31.771739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 1 16:13:31 2022] GET /identity/v3/role_assignments?user.id=f9fb5099a6c24abebce19b1a9dfe02b2&scope.domain.id=default&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:33.894723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59700d0-c3ad-4a2b-a424-da00e71545bc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:33.895256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59700d0-c3ad-4a2b-a424-da00e71545bc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:33.895332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59700d0-c3ad-4a2b-a424-da00e71545bc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:33.896948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:33.902608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b33cc-f297-488c-8903-3ed764126a31 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:33.903089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b33cc-f297-488c-8903-3ed764126a31 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:33.903198 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b33cc-f297-488c-8903-3ed764126a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:33.933691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-544b33cc-f297-488c-8903-3ed764126a31 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:33.934887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-544b33cc-f297-488c-8903-3ed764126a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:34.005784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9174-e87f-48f3-9f3b-2d84a9a222bf None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.006126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9174-e87f-48f3-9f3b-2d84a9a222bf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.006439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c9174-e87f-48f3-9f3b-2d84a9a222bf None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.008131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 12/23] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:13:34.013206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3439-21a7-4ab2-b7ad-73e2e20e5e9d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.013522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3439-21a7-4ab2-b7ad-73e2e20e5e9d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.013768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3439-21a7-4ab2-b7ad-73e2e20e5e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.040523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 12/24] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:34.056184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1bed3439-21a7-4ab2-b7ad-73e2e20e5e9d None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:34.057480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1bed3439-21a7-4ab2-b7ad-73e2e20e5e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:34.153831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:34.590293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6500b-0576-4354-a6ac-664f53d45e6d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.590589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6500b-0576-4354-a6ac-664f53d45e6d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.590841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6500b-0576-4354-a6ac-664f53d45e6d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.592379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 13/26] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:34.597850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.598140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.598416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.604981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6500b-0576-4354-a6ac-664f53d45e6d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:34.638002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:34.640858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:34.719568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:34.753665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:34.767519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 14/27] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:34.768451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:34.816787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6500b-0576-4354-a6ac-664f53d45e6d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:34.883881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:34.884491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.884769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.885044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.905904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5541d83e-6ea7-4d06-8fde-e441aad2c38b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:34.914407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:34.915318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:34.916070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 14/28] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:34.919849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa257a-ffdf-43e3-a681-0fca55efd3af None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.919849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa257a-ffdf-43e3-a681-0fca55efd3af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.919849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa257a-ffdf-43e3-a681-0fca55efd3af None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.919849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 15/29] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:34.923989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:34.924258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:34.924536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:34.954105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:34.956900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:34.959207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c4588-4538-4ed1-8023-c1d6c9aaf431 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.017631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.018616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.019129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.019734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.032456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.034592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:13:35.058919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:35.059876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:13:35.061149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:13:35.061149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:13:35.061149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:35.067033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 15/30] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:35.067983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.091759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 16/31] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:35.093189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.099658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:35.163622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:35.205754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b0fcec-c69d-4716-be93-71ece08f6062 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.227128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3c59c-229c-490d-88ff-6816d943a348 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.251173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.251173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.251173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.251173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.251493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:35.251493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.252818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.274833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 17/32] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:35.275717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.304765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.305385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.305668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.306046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.306828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:35.307134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.308543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.323002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 16/33] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:35.323882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.393215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da6a0a-ed39-40c9-ba03-149e320b2ff6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.423614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:35.431439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.432055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.432331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.432598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.442838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.442838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.463568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:35.473677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc0b1a-a442-4d0d-87c4-f9c4ca99c991 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.508301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.508984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.509302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.509553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.510460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.512000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8dd95-8022-49e2-8170-72a9c74d4760 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.668709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 18/34] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 16:13:35 2022] POST /identity/v3/roles => generated 219 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:35.669781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.750288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 17/35] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:13:35 2022] POST /identity/v3/projects => generated 301 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:35.753908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9455-844e-4ed4-a2db-716a59c51648 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.753908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9455-844e-4ed4-a2db-716a59c51648 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.753908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9455-844e-4ed4-a2db-716a59c51648 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.753908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 18/36] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:35.754493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9455-844e-4ed4-a2db-716a59c51648 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:35.904935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9455-844e-4ed4-a2db-716a59c51648 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.927609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f03b5-b46f-4601-a917-ebb0505fac25 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:35.950227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.951318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.951608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.951888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.952574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.954092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18e644-e9b3-4996-b540-f235368446dc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.965690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:35.966412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.972537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.972537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.972537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:35.972537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630ceb0-6889-44c2-8ed9-4003c92c6ccf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:35.972537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 19/37] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:35 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:35.972859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c94f3-646e-4bd6-b447-cd0ed78e3eef None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.973092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c94f3-646e-4bd6-b447-cd0ed78e3eef None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.973286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c94f3-646e-4bd6-b447-cd0ed78e3eef None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.974710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 20/38] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:35.976473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:35.976473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:35.976473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:35.994961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:35.996519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:36.042503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:36.062613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 21/39] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:36.063513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:36.076863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 19/40] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 16:13:35 2022] POST /identity/v3/roles => generated 225 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:36.078762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.079126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.079536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.087704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:36.116271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:36.118104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:36.199055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5a807-1066-48fc-ba87-5683b9031a04 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:36.220622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 20/41] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:36.243454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:36.244149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.244432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.244706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.245913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:36.247478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:36.249357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 22/42] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:36.254974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:36.342741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:36.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ddba96-3400-4e83-9bae-f1de791dbab4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:36.395523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:36.402167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:36.402302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:36.403082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.403408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.410007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] PATH_INFO: `/v3/domains/default/users/f9fb5099a6c24abebce19b1a9dfe02b2/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.410007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f9fb5099a6c24abebce19b1a9dfe02b2, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:36.410007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:36.453465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fa555-278e-4a1a-ad53-7e39053df8d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:36.488164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:36.488783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.489060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.489422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.490227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:36.491969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:36.543266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Sep 1 16:13:36 2022] PUT /identity/v3/domains/default/users/f9fb5099a6c24abebce19b1a9dfe02b2/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:36.544093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:36.632732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 16:13:36 2022] POST /identity/v3/roles => generated 223 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:13:36.661348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af79d5-4986-4523-9a61-8433192e2508 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:36.704340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:36.705243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.705702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.705979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.707233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:36.708725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62255ab1-1515-4e2d-bde2-9de751a7422a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:36.710578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 22/45] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:36.716727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:36.829571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07e880-962b-41ca-905a-d1061d882b30 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:36.840869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:36.866604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:36.867304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:36.867590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:36.867864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:36.868535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:36.870033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad8f9a-8aa3-436e-8308-11f2238696e3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:36.960741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:13:36 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:38.462070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4abe-000b-432a-b208-ccbc9ecb8d6f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:38.462606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4abe-000b-432a-b208-ccbc9ecb8d6f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:38.462606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4abe-000b-432a-b208-ccbc9ecb8d6f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:38.463983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 23/47] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:38.469182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8cd45-f149-49dc-bfaf-4dd138137220 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:38.469903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8cd45-f149-49dc-bfaf-4dd138137220 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:38.470183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8cd45-f149-49dc-bfaf-4dd138137220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:38.488637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-63c8cd45-f149-49dc-bfaf-4dd138137220 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:38.489825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63c8cd45-f149-49dc-bfaf-4dd138137220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:38.556452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 25/48] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:38.823461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d4abe-000b-432a-b208-ccbc9ecb8d6f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:38.878427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dd33-8053-4336-8a8a-0ca8dc82284b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:38.878708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dd33-8053-4336-8a8a-0ca8dc82284b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:38.878985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b4dd33-8053-4336-8a8a-0ca8dc82284b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:38.880387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 26/49] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:38.885489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:38.886137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:38.886137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:38.892390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:38.904433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:38.905634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:38.936600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d4abe-000b-432a-b208-ccbc9ecb8d6f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:38.970721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:38.971384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:38.971666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:38.971947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:38.973888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 27/50] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:38.984366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:38.990489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:38.991373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:38.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 24/51] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:38.997442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.020993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:39.095451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752446d-53b5-42a9-abb7-bc0b75cdfb85 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.128180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.128785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.129063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.129334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.129995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:39.130293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.131741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.139666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 28/52] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:39.145316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.244918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e54ce-a037-455d-b153-c18b5e2d18fc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.248029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.266449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:39.280588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.281223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.281503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.281776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.282918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.284424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.286202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 25/53] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:39 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:39.292447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.335661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:39.351546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae71e6f-afdb-48fc-b6d0-81a1d093c1a6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.386501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.387185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.387630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.387908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.389038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.390511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.392219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 29/54] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:39.393430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc007ad5-755b-4bea-a970-7fe3fa08bcc5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.397848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.428300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.429319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.429593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.429863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.430975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.432493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.434092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 26/55] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:39.439746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:39.496920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9569db3c-c9a7-4f9b-b5ce-fbaa5cae7439 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.497425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:39.530756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.531416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.531696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.531966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.532636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.534137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0238f6-5746-4db7-85cb-0c7c2c091263 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.653230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 30/56] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 1 16:13:39 2022] POST /identity/v3/projects => generated 298 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:13:39.658838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e6e36e-e35d-482e-83f7-26384656c634 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:39.697278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:39.697908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:39.698195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:39.698475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:39.699275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.domain.id=default, user.id=f9fb5099a6c24abebce19b1a9dfe02b2) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:39.699587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:39.701080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee0130-a89b-454a-be23-1e10e62e14a4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:39.710874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 27/57] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Sep 1 16:13:39 2022] GET /identity/v3/role_assignments?user.id=f9fb5099a6c24abebce19b1a9dfe02b2&scope.domain.id=default&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 530 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:41.596758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ae23-d3fe-4cab-8ce1-f6f87389aa13 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:41.597270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ae23-d3fe-4cab-8ce1-f6f87389aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:41.597355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6ae23-d3fe-4cab-8ce1-f6f87389aa13 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:41.598654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 31/58] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:41.604221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:41.604661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:41.604856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:41.622640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:41.623884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:41.695474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 28/59] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:41.970076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6ae23-d3fe-4cab-8ce1-f6f87389aa13 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:41.973892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:42.072694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6ae23-d3fe-4cab-8ce1-f6f87389aa13 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:42.104940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:42.105561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:42.105838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:42.106108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:42.107227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:42.108689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24785a0-6631-4eb9-8169-6c7fcda52676 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:42.110250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 32/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:42.116318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:42.164979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:42.213372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d4856-be8b-4973-853b-8e5a0d8197a1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:42.246487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:42.247177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:42.247466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:42.247735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:42.248394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:42.249828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fc78c-814a-400a-9460-8ff154407c6e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:42.364075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 29/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:13:42 2022] POST /identity/v3/users => generated 275 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:45.507865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ab72-b6d6-49c1-a44f-ba3a60328b44 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.508846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ab72-b6d6-49c1-a44f-ba3a60328b44 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.509349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10ab72-b6d6-49c1-a44f-ba3a60328b44 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.510903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 33/62] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:45.516255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b93408-6c15-4b06-b5c9-a1b58dc27e5b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.516797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b93408-6c15-4b06-b5c9-a1b58dc27e5b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.517289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b93408-6c15-4b06-b5c9-a1b58dc27e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.524665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad85-3bc2-4eac-9690-4c984e7b5823 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.525554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad85-3bc2-4eac-9690-4c984e7b5823 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.526026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ad85-3bc2-4eac-9690-4c984e7b5823 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.527677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 30/63] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:45.537483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.538072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.538566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.562983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c3b93408-6c15-4b06-b5c9-a1b58dc27e5b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:45.564229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3b93408-6c15-4b06-b5c9-a1b58dc27e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:45.571605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:45.572900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:45.700091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 34/64] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:45.702548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557081-7852-4fea-85d7-e1ffddeaaea1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.703248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557081-7852-4fea-85d7-e1ffddeaaea1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.705765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29557081-7852-4fea-85d7-e1ffddeaaea1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.707328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 35/65] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:45.708887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 31/66] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:45.716919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:45.717227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:45.717509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:45.733016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:45.753177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:45.755630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:45.885438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 36/67] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:45.976858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.032295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.120189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0917e8-1dd4-43e2-8213-18ae6a3b4f86 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.142814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.193499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.194066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.194349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.194620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.209969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-dcaa6b20-9875-421c-88b3-793dba588e98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:46.215501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:46.216369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:46.217109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:46.219710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 32/68] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:46.221673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ba75f-6025-4451-acc1-a647b81115b3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.222103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ba75f-6025-4451-acc1-a647b81115b3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.222587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ba75f-6025-4451-acc1-a647b81115b3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.224247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 33/69] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:46.225180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ba75f-6025-4451-acc1-a647b81115b3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.262124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb87cab-bfd9-4bdd-9717-4a598b09960d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.292377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.297204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.297851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.298140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.298420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.309635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d11d6c16-7b4c-4158-9e18-ccc34d4c9ca1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:46.314732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:46.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:46.317747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 37/70] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:13:46.321167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e627-100e-4991-bf03-a735b70d5cba None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.321615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e627-100e-4991-bf03-a735b70d5cba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.321891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1e627-100e-4991-bf03-a735b70d5cba None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.323217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 38/71] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:46.324076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1e627-100e-4991-bf03-a735b70d5cba None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.326840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.333330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ba75f-6025-4451-acc1-a647b81115b3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.368758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.369391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.369665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.369935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.381066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0b5fef05-81c4-4213-ae26-e23729b61bcc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:46.385877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:46.386913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:46.387752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 34/72] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:46.388372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.388442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.388712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.406203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:46.407377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:46.431616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1e627-100e-4991-bf03-a735b70d5cba None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.472747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.473466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.473960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.474240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.474626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 35/73] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:46.486470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:46.486470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.486650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.486728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e47bc7-9eab-4b28-8227-27d38e3bb6f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.486728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 39/74] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.486728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.486930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.487080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.507114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:46.508601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:46.561898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.568327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.581174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 40/75] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:46.591336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.593941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d77f372-c031-427e-8886-fabdee9ecf87 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.633852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.634470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.634746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.635019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.635729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:46.636029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.637488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.646751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 36/76] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.647582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.709340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ed488-4d01-428b-a5ba-26d3508b4626 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.747396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.748014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.748296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.748567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.749235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:46.749543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.750934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.759286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 41/77] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.764117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.774025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca663e-d134-4ed2-8201-3e6a54c765ed None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.809851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.810500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.810783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.811053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.812214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.813649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.815398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 37/78] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.816083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.816327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.838364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:46.877201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d8ec7-7bae-4522-b38a-272768e2c18c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.912847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.913484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.913766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.914044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.920710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c8c6d-2eb7-4681-9312-95943333d3dd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:46.927572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.929280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.930888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 42/79] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.931686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:46.961132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:46.961691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:46.961971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:46.962247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:46.963408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:46.964870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:46.966442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 38/80] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:46.968012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.029591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853da96-5b57-423e-81ca-c0b71bbcde77 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.031897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.050799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.064904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.065616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.066097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.066546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.067505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb959f-6259-432f-8826-d329f52f6d61 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.068087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.069559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.071143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 43/81] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:47.071942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.101932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.102549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.102830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.103108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.170841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.172349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.172503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120bf251-9d63-4072-99e2-a95ccb2d4210 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.206357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.206988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.207304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.208165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.218482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:47.223920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:47.224930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:47.225820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 44/82] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:47.226449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.257179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.261722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.278226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 39/83] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:13:47.279036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.330654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c950-1988-42e7-9dee-5e37844cfe43 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.366861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.367507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.367785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.368058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.378686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18c627-8381-4aff-94f2-3582395c2931 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.412986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.413591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.413869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.414144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.414797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.416277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.437933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.439483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.469443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.507960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 40/84] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:13:47 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:47.508828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.546747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 45/85] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:13:47.546747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.562793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.640540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ab74a-b3ad-4c74-b2cc-5685e6e44f8f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.686173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.686800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.687080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.687397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.696790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b32296-8c61-444c-9402-9685f8d3f6b2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.738696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.743282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.743910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.744190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.744472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.745236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:47.745876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.746930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.755835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 46/86] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:47.756738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.771789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.772869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.822551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:47.889204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 41/87] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:13:47.890071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:47.899451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09b5b1-09f7-4b87-bbc4-d46d7f7e3bb0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:47.948616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:47.949999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:47.950294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:47.950691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:47.952247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:47.954426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:47.957838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 47/88] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:47.958485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.008924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c229d4a-0138-4a65-a810-5f8adc8f5a40 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.032583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.057798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.060845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.060845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.060845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.060845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.062779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.065238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 42/89] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.066150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.079557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4434-6677-4a10-a11f-3b7740b7d3c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.079732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.116066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.117788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.118177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.118558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.119985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.121647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.123853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 48/90] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:47 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.124968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.193233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecff0b-8dec-407b-b34c-753dab98a968 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.236514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.236721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e71d87b-0720-454a-b190-b16c44993d30 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.237154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.237433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.237704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.239502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.241962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.246739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 43/91] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.247495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.275815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.282908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.283933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.284364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.284784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.286325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=f9fb5099a6c24abebce19b1a9dfe02b2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:48.295385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.295385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.295385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 49/92] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/role_assignments?user.id=f9fb5099a6c24abebce19b1a9dfe02b2&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.296167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.354769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.413539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18e9b5-be94-44c6-8f6e-c34a0d0d9672 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.457801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.458570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.458849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.459123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.460045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:48.460356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.461785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.467895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 44/93] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.468787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.475110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166d828-4a64-49c1-8498-9ce4ed63c4af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.535919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.536603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.536977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.537255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.654713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.663557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7836620-131a-45b5-b39f-3684424ef616 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:48.673818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.675331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.735684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:48.736312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:48.736592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:48.736868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:48.737599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:48.737906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:48.741744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:48.756464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 45/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=8488c378257a4f629184ae9773b882ac => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:48.842616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 50/95] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:13:48.848691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:48.852848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:48.977793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2493cf-3a03-417d-b415-df457250df9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:49.016190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:49.016844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:49.017122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:49.017401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] PATH_INFO: `/v3/projects/7002059b4b62482b98931855bcbf8f00` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:49.018658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] RBAC: Authorizing `identity:get_project(project_id=7002059b4b62482b98931855bcbf8f00)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:49.020909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a7bb7-1409-4add-9483-4d9408d85e52 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:49.022841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 46/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:48 2022] GET /identity/v3/projects/7002059b4b62482b98931855bcbf8f00 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:49.028767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:49.138818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:49.146947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da3a9-4648-44bf-8cd9-9c112910c09c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:49.186154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:49.186765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:49.187045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:49.187342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:49.188062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=7002059b4b62482b98931855bcbf8f00, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:49.188378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:49.189884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f02243-639d-4bca-a108-b337fc9415a2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:49.199401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 51/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:49 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=7002059b4b62482b98931855bcbf8f00&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:50.146654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b810-0598-4c6a-bb08-20187e20d727 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.147213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b810-0598-4c6a-bb08-20187e20d727 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.147213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f306b810-0598-4c6a-bb08-20187e20d727 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.148628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 47/98] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:13:50.153999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049051-3d82-4ec7-9e39-bae916ed961e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.154395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049051-3d82-4ec7-9e39-bae916ed961e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.154578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08049051-3d82-4ec7-9e39-bae916ed961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.179701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-08049051-3d82-4ec7-9e39-bae916ed961e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:50.180941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08049051-3d82-4ec7-9e39-bae916ed961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:50.222147 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208926cd-3ec9-49ef-90c1-11d1175e0279 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.222444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208926cd-3ec9-49ef-90c1-11d1175e0279 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.222752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208926cd-3ec9-49ef-90c1-11d1175e0279 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.224363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 48/99] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:50.230059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697194c-d40a-48ef-9747-b738387b390e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.230396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697194c-d40a-48ef-9747-b738387b390e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.230666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697194c-d40a-48ef-9747-b738387b390e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.258891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d697194c-d40a-48ef-9747-b738387b390e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:50.262102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d697194c-d40a-48ef-9747-b738387b390e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:50.307371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 52/100] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:50.354970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 49/101] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:13:50.678736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ce4-767b-4d17-a861-f57bee0d77b5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.679123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ce4-767b-4d17-a861-f57bee0d77b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.679329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013ce4-767b-4d17-a861-f57bee0d77b5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.680679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 53/102] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:50.685889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:50.686176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:50.686452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:50.693285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:50.709977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:50.711182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:50.802614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 50/103] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:50.831240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013ce4-767b-4d17-a861-f57bee0d77b5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:50.857963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:50.963267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c846d38-279d-4653-a78b-cfc74c9271a6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:50.970318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:50.995224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:51.000835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.001469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.001746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.002017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.003861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:51.006292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:51.008327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 51/104] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:51.009549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013ce4-767b-4d17-a861-f57bee0d77b5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.017934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.091870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.092765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.093212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.093619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.119719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:51.124821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:51.125849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:13:51.126605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:51.128763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 54/105] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:51.140125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ddaa9-78d6-4f79-984f-fa206c08b3fc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.148917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.181274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.182062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.182411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.182775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.183946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:51.185780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:51.283584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 52/106] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:13:51 2022] POST /identity/v3/projects => generated 302 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:51.297205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7d6a1-f620-46e3-b95d-35161a8e6303 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.337724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.368570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.369216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.369505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.369779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.370477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:51.370783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:51.372282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:51.390846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 55/107] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:51.404788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.435269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:51.442646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:51.501394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61a903-f61c-4918-809f-071c5216a3bd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.536596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.541745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.541745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.541745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.548240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:51.553813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:51.554679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:51.555560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 53/108] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:51.560625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.655676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa44f2-fa3c-4f46-b3f8-ad5db55b6537 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.714640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669eac7-6a39-4b18-bf4b-67ce84ec35b0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.719986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.720632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.720909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.721176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] PATH_INFO: `/v3/users/d967a26018414719b527f8265c92ccb1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.743700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:51.751196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=d967a26018414719b527f8265c92ccb1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:51.752764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6a59e-725f-4c98-ad49-cadeb962f9f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:51.755750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 56/109] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/users/d967a26018414719b527f8265c92ccb1 => generated 289 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:51.756911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0597-58fb-415c-9d20-c8cfa679576e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.757260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0597-58fb-415c-9d20-c8cfa679576e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.757456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb0597-58fb-415c-9d20-c8cfa679576e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.762481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 57/110] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:51.762481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb0597-58fb-415c-9d20-c8cfa679576e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.781571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.782270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.782466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.782740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.783444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:51.783798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:51.785489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab08e-37f5-4132-b5c3-fa7a18fba763 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:51.795515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 54/111] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:51.796840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.802756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.803050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:51.835662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:51.836886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:51.857960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:51.914675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb0597-58fb-415c-9d20-c8cfa679576e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:51.918241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 55/112] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:51.919232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:51.961425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:51.962043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:51.962322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:51.962597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.098267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.100449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.120953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d6089-9415-4dfe-99d5-5cf349a2dd00 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.124870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.159607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.160273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.160553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.161021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.162330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.163819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cce2-fd7f-4f4b-a381-f4bd4bb30195 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.165586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 56/113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:52.166559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135862-2737-49d8-be83-86ea3a27fbaa None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.166853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135862-2737-49d8-be83-86ea3a27fbaa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.167130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135862-2737-49d8-be83-86ea3a27fbaa None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.168398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 57/114] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:52.180570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.180854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.181122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.210311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:52.211558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:52.232284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.275024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 58/115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:13:52.281109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.295766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 58/116] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:52.301060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.417087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c1926-02bb-411c-9413-a154f0951326 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.423662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d0aea-9b23-4d2c-b072-836588e72dbb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.425480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.458384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.459373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.459793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.460324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.460487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.460964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.461248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.461522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.462301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.463995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.465974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 59/117] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:52.466986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.472535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8c450865-11bb-4091-9c25-990c0b03423a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:52.478057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:52.479071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:52.480240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 59/118] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:52.484745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b665-19e7-4467-b910-4594eecb24c7 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.484961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b665-19e7-4467-b910-4594eecb24c7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.485240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782b665-19e7-4467-b910-4594eecb24c7 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.491179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 60/119] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:52.491179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b665-19e7-4467-b910-4594eecb24c7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.563548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.587238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05606995-9fa6-4b17-9149-2d418130d540 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.599999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782b665-19e7-4467-b910-4594eecb24c7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.625763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.626397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.626674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.626945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.628697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.631359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.633347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 60/120] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:52.636876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.637537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.637810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.638168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/users/f9fb5099a6c24abebce19b1a9dfe02b2/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.639057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.640117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, user_id=f9fb5099a6c24abebce19b1a9dfe02b2, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.641565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e22b6-2210-4117-bf0e-19b6a9a1007b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.711827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 61/121] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:13:52 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/users/f9fb5099a6c24abebce19b1a9dfe02b2/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:52.713194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.713484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.713754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.733864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:52.735355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:52.763849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.766881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9687790e-dc33-4477-9955-96d145bd5e61 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:52.771251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:52.824332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:52.825314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:52.825671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:52.826169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:52.826934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:52.827672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:52.829453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:52.832673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 62/122] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:52.833974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.856986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 61/123] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:52.865204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:52.993938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442375e-b79d-4179-92f4-8c8381eae658 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.009840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addeabd5-e808-494c-b825-e5b7d234c0ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.040207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.040853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.041133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.041557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.054337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:53.058869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:53.059460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:53.060305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:53.061062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 63/124] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:53.061684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.062887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.063470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.063932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.064917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=d967a26018414719b527f8265c92ccb1) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:53.065426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.072364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.081056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 62/125] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:52 2022] GET /identity/v3/role_assignments?user.id=d967a26018414719b527f8265c92ccb1&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=fc5855d774c84359acf01fccb8000531 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.082703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.093191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.149528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.277248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8585a3a-2908-4798-99a1-600dc4fe0379 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.300708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c55a4-4936-4274-9af6-ab38272a0510 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.330899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.332504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.332989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.333504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.334890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.337256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.339428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 64/126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.340435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.341134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.341752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.342032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.342300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.343038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:53.343417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.345146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.356254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 63/127] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.357036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.466748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf652-f3b7-4dc6-bbe8-d0b9c03b391c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.487597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.504581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.505274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.505707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.506055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.508081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.509562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.511236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 64/128] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.512419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.547497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.597196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe01712-9187-4709-9bf8-5a4b04ff5e4a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.625338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea2010-ee58-45ad-9670-bfc0b6d9c0db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.663645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.664609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.664890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.665165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.666395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.667990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.669733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 65/129] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.670983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.688759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.689535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.689820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.690116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.701430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:53.705001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2375b804-6ce6-471e-8d9e-1c548635af14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:53.708647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:53.709674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:53.710626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:53.712927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 65/130] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:53.714266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6887-a67a-49f1-8c0c-b655f23602af None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.714692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6887-a67a-49f1-8c0c-b655f23602af None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.715106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb6887-a67a-49f1-8c0c-b655f23602af None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.720634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 66/131] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:53.721713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb6887-a67a-49f1-8c0c-b655f23602af None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:53.800264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c4dbf-f318-42cc-98ff-94a8c180f261 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.838607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.838827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb6887-a67a-49f1-8c0c-b655f23602af None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:53.839869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.840434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.841021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/users/3cbff81e600346ad949641c0f50e4198/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.843213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, user_id=3cbff81e600346ad949641c0f50e4198, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.844962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.876041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:53.876683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.876955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.877223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.878331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:53.880328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d1ae62-91f7-4c99-a2af-149a1efef30a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:53.882121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 67/132] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:53.883457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:53.883736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:53.884001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:53.911831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:53.913031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:53.915003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:53.925340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 66/133] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:13:53 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/users/3cbff81e600346ad949641c0f50e4198/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:53.926995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.037761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 68/134] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:54.046303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.123588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:54.150718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f4ff6d-a29c-441f-8fcf-d82d398cc44a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.214138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.215329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.216857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.218134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.219573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:54.220148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.221931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.237998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 67/135] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:54.240490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a40f29-9ed9-4b4f-904c-f6b7d030cea0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.246765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.303951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.304623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.305133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.305437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/users/3cbff81e600346ad949641c0f50e4198/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.307700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, user_id=3cbff81e600346ad949641c0f50e4198, role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.309659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.404122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:54.409448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 69/136] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:13:54 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/users/3cbff81e600346ad949641c0f50e4198/roles/8488c378257a4f629184ae9773b882ac => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:54.410390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.451434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b272e87-3ad2-4aec-8ac0-3c91cc604f1a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.520738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.521383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.521657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.521928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.524139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.525605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.527217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 68/137] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:54 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:54.535402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.573585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:54.610992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47659c00-6182-474b-939e-bb69ecf367be None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.684591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.685377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.685662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.687202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.689109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.690594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.695910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 70/138] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:13:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:54.702388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.724883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd429ba-2be6-4119-b84e-ff5a25cd30e6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.791844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.792494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.792773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.793050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] PATH_INFO: `/v3/projects/7002059b4b62482b98931855bcbf8f00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.795363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] RBAC: Authorizing `identity:get_project(project_id=7002059b4b62482b98931855bcbf8f00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.796841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.799592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 69/139] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:54 2022] GET /identity/v3/projects/7002059b4b62482b98931855bcbf8f00 => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:54.806590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:54.837757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:54.903302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8ee92-7ade-4a05-8878-1d278b91cce5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:54.968324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:54.969353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:54.970422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:54.970715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:54.970991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:54.971698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:54.973911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec6ec8b-6cdd-4080-990f-78fac5586a97 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:54.996297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793f96-9d71-4158-9b93-21b2f64e43f8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:55.037786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:55.038512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:55.038804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:55.039082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] PATH_INFO: `/v3/projects/7002059b4b62482b98931855bcbf8f00/users/3cbff81e600346ad949641c0f50e4198/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:55.041066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7002059b4b62482b98931855bcbf8f00, user_id=3cbff81e600346ad949641c0f50e4198, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:55.042537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e0cc1-f7da-402c-a985-9a83ac4037e7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:55.105688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 71/140] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:13:54 2022] POST /identity/v3/users => generated 283 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:13:55.165635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 70/141] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:13:54 2022] PUT /identity/v3/projects/7002059b4b62482b98931855bcbf8f00/users/3cbff81e600346ad949641c0f50e4198/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:55.407615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b38c7-f5b8-430c-a6a0-6d5de4a9643b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:55.408114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b38c7-f5b8-430c-a6a0-6d5de4a9643b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:55.408281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b38c7-f5b8-430c-a6a0-6d5de4a9643b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:55.409546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 72/142] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:55.416054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99a99-9f2f-4e02-8d15-0c4a0eaf9ef9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:55.416344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99a99-9f2f-4e02-8d15-0c4a0eaf9ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:55.416634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99a99-9f2f-4e02-8d15-0c4a0eaf9ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:55.438502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e4f99a99-9f2f-4e02-8d15-0c4a0eaf9ef9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:55.439744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e4f99a99-9f2f-4e02-8d15-0c4a0eaf9ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:55.524058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 71/143] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:55.538056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754151-1c28-4075-982b-c206a9aa49b3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:55.538347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754151-1c28-4075-982b-c206a9aa49b3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:55.538631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f754151-1c28-4075-982b-c206a9aa49b3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:55.540172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 73/144] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:55.545402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:55.545694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:55.545967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:55.566449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:55.567697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:55.641009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 72/145] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:55.942742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f754151-1c28-4075-982b-c206a9aa49b3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.063427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f754151-1c28-4075-982b-c206a9aa49b3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.069318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.098473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:56.100761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:56.124378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.125018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.125292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.125560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.149265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:56.153261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:56.154102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:56.154855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 74/146] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:56.162003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.278067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7341e7-131e-42a1-853d-7fd8e10f80e6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.338172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.338885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.339177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.339478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.353039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:56.357151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab621c-91d5-40a1-9574-c82e7e8dccaf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:56.358090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:56.358918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:13:56.359707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 73/147] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:13:56.365955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.447545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:56.454189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.454810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.455094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.455406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.456223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:56.457015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:56.458486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:56.473160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 75/148] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:56.478384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.480379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79444c7-e871-43d2-a257-c4fd861cf869 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.518697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.519923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.522164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.522164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.522518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:56.522823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:56.525839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:56.535640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 74/149] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:56.541527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.559316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:56.654915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bf5be-c0fe-4b3f-bdf1-a0599a9cdaf1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.687557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:56.695161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.695822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.696101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.696420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] PATH_INFO: `/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.700330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] RBAC: Authorizing `identity:get_user(user_id=f9fb5099a6c24abebce19b1a9dfe02b2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:56.701847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:56.703703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 75/150] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/users/f9fb5099a6c24abebce19b1a9dfe02b2 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:56.711580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.724196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f9258-215d-4a9d-9b7e-4672090fd086 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.833488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.834137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.834412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.834683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] PATH_INFO: `/v3/users/d967a26018414719b527f8265c92ccb1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:56.835862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] RBAC: Authorizing `identity:get_user(user_id=d967a26018414719b527f8265c92ccb1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:56.837358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:56.839267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 76/151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/users/d967a26018414719b527f8265c92ccb1 => generated 289 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:56.845190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:56.900897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d19437-2c41-4c0e-9208-0eff692050ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:56.969044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:56.969681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:56.969963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:56.970236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.110379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:57.112396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:57.114128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3ddda-a7e1-43f9-9b98-bc5dc1836695 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:57.141173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e3c07-df6e-4a1e-a78e-da398b0898d4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:57.171286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:57.232800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:57.233454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.233730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.234003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.235171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:57.236626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ff4d4-815e-4cd4-9b23-ed262d5f18eb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:57.238356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 77/152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:57.239649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27094d9-0a26-44e3-986d-968f6e81bf6f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.239928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27094d9-0a26-44e3-986d-968f6e81bf6f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.240198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27094d9-0a26-44e3-986d-968f6e81bf6f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.248973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 78/153] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:57.249771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27094d9-0a26-44e3-986d-968f6e81bf6f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:57.327699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 76/154] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:13:57.328796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.329429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.329709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.351680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:57.353270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:57.428610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 77/155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:57.429938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:57.460368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:57.494053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27094d9-0a26-44e3-986d-968f6e81bf6f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:57.541145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d289d-0a13-4c5b-b4ea-6532cf6f574a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:57.578294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:57.579068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.579376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.579731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/users/d967a26018414719b527f8265c92ccb1/roles/fc5855d774c84359acf01fccb8000531` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.580984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:57.581718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, user_id=d967a26018414719b527f8265c92ccb1, role_id=fc5855d774c84359acf01fccb8000531)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:57.582559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.582996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.583188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08813b49-e501-4c69-9379-0d970019a87c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:57.583596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.586455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:57.586866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:57.592000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 78/156] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:57 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:57.595490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:57.700960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:57.708929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea0734-011f-4b4b-985b-27309e1304f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:57.723872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 79/157] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:13:57 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/users/d967a26018414719b527f8265c92ccb1/roles/fc5855d774c84359acf01fccb8000531 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:13:57.747814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:57.748773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.749204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.749660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.750559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=f9fb5099a6c24abebce19b1a9dfe02b2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:57.751019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:57.753060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6d19fc-c460-4aa8-97af-b5bbf1145842 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:57.762981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 79/158] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:13:57 2022] GET /identity/v3/role_assignments?user.id=f9fb5099a6c24abebce19b1a9dfe02b2&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:57.848306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2090-014d-45af-8106-8357c9e121f3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.848752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2090-014d-45af-8106-8357c9e121f3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.849038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a2090-014d-45af-8106-8357c9e121f3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.850760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 80/159] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:57.856601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:57.857200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:57.857704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:57.883761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:57.885339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:57.972870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 80/160] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:58.186044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a2090-014d-45af-8106-8357c9e121f3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.240784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:58.310725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a2090-014d-45af-8106-8357c9e121f3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:58.352392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:58.353136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:58.353409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:58.353675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:58.366469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:58.371451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:58.372434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:58.373270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 81/161] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:58.380711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.479839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:58.494205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568d3dc-874e-4e61-9538-1d04fbc0ac08 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:58.533431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:58.534050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:58.534884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:58.535190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:58.535889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:58.536243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:58.537761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:58.548753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 81/162] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:58.554393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.563845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.766996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df01105-2b22-45c6-8a47-a4d6e6c11160 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:58.770407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:58.811177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba60385-73ba-4a8b-9008-b3db455c7a22 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:58.856863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:58.857514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:58.857794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:58.858065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:58.896125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:13:58.900720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:13:58.901089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:13:58.901929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:13:58.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:13:58.903308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:58.903308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:58.903308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:58.903742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 82/163] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:13:58.905057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:58.906493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:58.908081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 82/164] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:58.911762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.914360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:58.964296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:59.148277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812847ef-3b54-4131-bcde-20f4d590adad None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:59.151778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956263b-ed7b-4a06-8f13-d4cc8c3b2491 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:59.209952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:59.210554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.210826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.211091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.211783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:13:59.212078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:59.213463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:59.215306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:59.215911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.216180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.216441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.231734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 83/165] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:13:59.237458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:59.278553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:59.341080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:59.342558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a61bd-e954-4d85-ae07-ee4614048dd9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:59.377144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:59.413801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbfde3-f6a5-4856-91df-f38fc6d9ef64 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:59.478066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:59.478701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.478974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.479259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.481505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:59.483173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f87e27-6303-4e5b-bdbe-9a97711500ab None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:59.484753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 84/166] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:13:59 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:13:59.485865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa044-05d6-4539-a632-e81862ade60d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.486139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa044-05d6-4539-a632-e81862ade60d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.486396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa044-05d6-4539-a632-e81862ade60d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.487955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 85/167] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:59.492387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fa044-05d6-4539-a632-e81862ade60d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:59.526863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 83/168] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:13:59.528052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b55-2b5c-4fb4-8488-c3355635cc85 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.528339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b55-2b5c-4fb4-8488-c3355635cc85 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.528610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e764b55-2b5c-4fb4-8488-c3355635cc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.546572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e764b55-2b5c-4fb4-8488-c3355635cc85 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:13:59.547775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e764b55-2b5c-4fb4-8488-c3355635cc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:13:59.666312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:59.675251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fa044-05d6-4539-a632-e81862ade60d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:13:59.720679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:13:59.721302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.721614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.721882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.729760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 84/169] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:13:59.731125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6652f-ba50-446d-b915-d60176fd3c39 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:13:59.731440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6652f-ba50-446d-b915-d60176fd3c39 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:13:59.731723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6652f-ba50-446d-b915-d60176fd3c39 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:13:59.733136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 85/170] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:13:59.733709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6652f-ba50-446d-b915-d60176fd3c39 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:13:59.839349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:13:59.840814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ad3ad-3d64-420e-8598-790eeeb6f825 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:13:59.894754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:13:59.996994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:00.011201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6652f-ba50-446d-b915-d60176fd3c39 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.011398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 86/171] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:00.015592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.015989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.016948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.045579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:00.046857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:00.108069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.109723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.110005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.110277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.115934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:00.117925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef817f-19df-414e-ad48-a310a7799732 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:00.119757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 86/172] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:13:59 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:00.121972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194af60-c652-4a8c-9d2a-aaf8701041b2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.122473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194af60-c652-4a8c-9d2a-aaf8701041b2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.122747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194af60-c652-4a8c-9d2a-aaf8701041b2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.124319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 87/173] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:00.126290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194af60-c652-4a8c-9d2a-aaf8701041b2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.201237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 87/174] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:00.202612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.348422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194af60-c652-4a8c-9d2a-aaf8701041b2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.366788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:00.387294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.387938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.388391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.388673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.389812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:00.391831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e60ade-7e97-44be-8108-96827e767593 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:00.394279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 88/175] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:00.395524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6af24-abe6-4c9e-9b9a-61df5f102826 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.395804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6af24-abe6-4c9e-9b9a-61df5f102826 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.396077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6af24-abe6-4c9e-9b9a-61df5f102826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.399338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955ce6-b52c-43ee-9f00-7279300dcd27 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.421838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4dc6af24-abe6-4c9e-9b9a-61df5f102826 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:00.423339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4dc6af24-abe6-4c9e-9b9a-61df5f102826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:00.428907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:00.444763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.445791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.446288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.446760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.457876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:00.464339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:00.465227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:00.466042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:00.468437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 88/176] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:00.474851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.501226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 89/177] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:00.502468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8639a-f961-44bb-b534-cf79dcda6d68 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.502768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8639a-f961-44bb-b534-cf79dcda6d68 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.503208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8639a-f961-44bb-b534-cf79dcda6d68 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.507330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 90/178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:00.508302 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8639a-f961-44bb-b534-cf79dcda6d68 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.590553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd83880-c2e4-43d2-aaba-6161db9ecf69 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.638464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.638986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.639288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.639555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.640282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:00.640582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:00.642004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:00.644660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8639a-f961-44bb-b534-cf79dcda6d68 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.655178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 89/179] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:00.655178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.680777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:00.719126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.719776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.720354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.720632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.723039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:00.723377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:00.724896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:00.734147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 91/180] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=8488c378257a4f629184ae9773b882ac => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:00.735028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.739965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:00.814978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b37ef-66cb-492f-87e6-a70293dfce64 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.865125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.866154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.867362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.867890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.869267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:00.869784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:00.872307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346206f-047a-439b-9c34-10e5898bad0b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:00.884337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 90/181] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:00.889061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.889061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.889241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.910986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:00.912213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:00.932797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4431d2a-241b-4592-8e36-4caef3a12026 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:00.971813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:00.972478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.972913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.973181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.984447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-39e3d470-5447-47e6-8072-ac163d086ebf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:00.990204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:00.991351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:00.992321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 92/182] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:00.993156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cfedf-8422-439a-a2c8-308d3861b971 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:00.993156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cfedf-8422-439a-a2c8-308d3861b971 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:00.993156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cfedf-8422-439a-a2c8-308d3861b971 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:00.994362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 93/183] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:00.994885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cfedf-8422-439a-a2c8-308d3861b971 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:00.996295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 91/184] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:00.997318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.038879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.039378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.103672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cfedf-8422-439a-a2c8-308d3861b971 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.104622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90399bc6-a91a-465a-8960-5662e6c14c3a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.139277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.139803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.140080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.140286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.140408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] PATH_INFO: `/v3/users/3cbff81e600346ad949641c0f50e4198` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.140908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.141180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.141447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.141553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] RBAC: Authorizing `identity:get_user(user_id=3cbff81e600346ad949641c0f50e4198)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.142530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.142890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205d42f-1823-420b-af7f-a0ccde3e5923 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.144352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.144580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 92/185] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/users/3cbff81e600346ad949641c0f50e4198 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:01.149736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.149845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 94/186] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:01.149922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.149922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.150060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.165595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:01.166782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:01.235078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 93/187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:01.235687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.280746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411496e-d176-4e2f-bd80-04438d808fe5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.288590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.289132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.318989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.320499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.320919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.321342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.322141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:01.322765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.325396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.337111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 95/188] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:01.338143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.360207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f3115-05bc-4187-b624-094aa3aced89 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.400580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.401447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.401859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.402258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.445689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191e04a-d6de-410d-8245-5038b9ab1a7f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.478346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.479356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.479968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.480243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.480513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.480678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.481161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.482586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.515203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.579895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 94/189] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:01.580756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.596548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 96/190] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:01 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:01.596548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.637842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.697235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeaf631-fa68-42e6-ba6d-68d531963e04 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.723459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fab399d-870a-4f73-97e5-44d672a322e6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.735180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.735831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.736118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.736391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.738373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.748429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:01.754323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:01.755373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:01.756169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:01.756865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:01.759185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.772247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.773475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.773937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.774364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.789482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.791019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.792735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 97/192] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:01.793586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.887684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70b519-c1d1-4358-b3ab-8a8f4e3d5644 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.900066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:01.919361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baffa9a-01c0-4804-a12d-3459cae6c6c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:01.931197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.931849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.932135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.932407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.945977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:01.951480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:01.952328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:01.953121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 96/193] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:01.954868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:01.970802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:01.971970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:01.972463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:01.972973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] PATH_INFO: `/v3/projects/7002059b4b62482b98931855bcbf8f00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:01.974545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] RBAC: Authorizing `identity:get_project(project_id=7002059b4b62482b98931855bcbf8f00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:01.976397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:01.978555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 98/194] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/projects/7002059b4b62482b98931855bcbf8f00 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:01.979576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.023982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.075882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684b075-3212-40bf-9be0-5a83dca6abc9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.093260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d282698-0efd-433e-99c6-e9549421b7de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.121439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.122140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.122423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.122695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.123391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:02.123695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.125094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.132765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.133668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.134142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.134269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 97/195] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.134773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.135115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.167503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.230694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.232311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.249842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a560ac-7448-402b-ae43-f8ab7607890b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.271695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.292624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915def-1649-468d-832d-8b1228878864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.293235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915def-1649-468d-832d-8b1228878864 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.294354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915def-1649-468d-832d-8b1228878864 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.294635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5915def-1649-468d-832d-8b1228878864 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.295414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915def-1649-468d-832d-8b1228878864 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:02.295673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915def-1649-468d-832d-8b1228878864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.297135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5915def-1649-468d-832d-8b1228878864 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.312504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 98/196] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.313519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915def-1649-468d-832d-8b1228878864 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.363016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 99/197] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:02.364427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.429425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5915def-1649-468d-832d-8b1228878864 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.431539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.467301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.467913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.468191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.468461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.469322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=7002059b4b62482b98931855bcbf8f00, user.id=3cbff81e600346ad949641c0f50e4198) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:02.470145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.472461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.478931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 99/198] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/role_assignments?user.id=3cbff81e600346ad949641c0f50e4198&scope.project.id=7002059b4b62482b98931855bcbf8f00&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.479840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.502316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3daf52-d9d5-466c-8f63-797d97fd24dc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.530492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.553951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.555010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.555536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.556167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.558190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.560325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.564360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 100/199] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.565635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.608697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fab7a-bca1-4d28-ba9e-8b47e1261271 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.657610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.659627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.660384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.660829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] PATH_INFO: `/v3/users/d967a26018414719b527f8265c92ccb1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.663045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d967a26018414719b527f8265c92ccb1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.664783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.667383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 100/200] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/users/d967a26018414719b527f8265c92ccb1 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.668370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.685114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb524748-4f39-402e-8ad2-c3487bd6e902 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.704950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.725062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.725863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.726291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.726639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.728392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.730016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:02.732308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 101/201] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:02.733480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:02.812701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075267b6-7484-4cb3-a365-03e06af6b9b7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.828112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:02.858995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.859800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.860225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.860502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.864307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594cc6-a136-4a7e-9fc8-89433a0c5447 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:02.920526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:02.921778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:02.922135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:02.922741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:02.950112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:02.951888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.003822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:03.024946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.027491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.070949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 101/202] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:03.072881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:03.091566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:03.153556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 102/203] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:03.154557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:03.199793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40fa005-cf36-4e58-9d4f-48bb0f7502da None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:03.239995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:03.240522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.240908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.241278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.242114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=32773fad843a4779ba2b5d258e24a241) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:03.242525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.244095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.250357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 102/204] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:03 2022] GET /identity/v3/role_assignments?user.id=32773fad843a4779ba2b5d258e24a241&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:03.251373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:03.259604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:03.280636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e921125-9fb2-46bf-bcba-d3a7c247f8c1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:03.352494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:03.353236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.353616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.353988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.357350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.359982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.366042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 103/205] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:03 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:03.373788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:03.444772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:03.483434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d4a7d-52ac-4e64-874b-cf9b5e20886e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:03.521392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:03.522393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.522654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.523040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.528241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.530008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.532159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 103/206] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:03 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:03.536342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45cc821-4d86-4437-81e2-e423f4e6e35d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:03.538605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:03.572720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:03.573710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.574193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.574674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.575627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=32773fad843a4779ba2b5d258e24a241) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:03.576146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.577795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10395806-5663-43d6-ab57-e328f28d360a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.584281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 104/207] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:03 2022] GET /identity/v3/role_assignments?user.id=32773fad843a4779ba2b5d258e24a241&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=8488c378257a4f629184ae9773b882ac => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:03.708290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e44149-8533-40e1-bda8-c9efd36202ff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:03.770464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:03.771084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.771388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.771660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.772401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, scope.project.id=bc0910ca971f4c5186644553060c147f, user.id=d967a26018414719b527f8265c92ccb1) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:03.772707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:03.774146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4725642-99d7-4d85-8422-393da4e52991 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:03.780708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 104/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:03 2022] GET /identity/v3/role_assignments?user.id=d967a26018414719b527f8265c92ccb1&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=fc5855d774c84359acf01fccb8000531 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:03.930832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45c71-75c9-4375-a852-f01415dbb62a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.931867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45c71-75c9-4375-a852-f01415dbb62a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.932340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e45c71-75c9-4375-a852-f01415dbb62a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.933906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 105/209] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:03.939612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.940705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.940705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.945209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f226-7e87-4c2a-8a04-07b2363dd30e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.945513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f226-7e87-4c2a-8a04-07b2363dd30e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.945750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583f226-7e87-4c2a-8a04-07b2363dd30e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.947382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 106/210] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:03.952637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:03.952915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:03.953192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:03.955368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:03.964063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:03.965229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:03.973855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:03.975041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:03.993676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:04.059255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 105/211] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:04.079326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 107/212] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:04.345062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.355661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.456199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2718a-00f5-443b-a940-2d750901227a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:04.476956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae3050c-e1aa-4c16-bf71-302ff87116a6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:04.494322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:04.495414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:04.495859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:04.496253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:04.497382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:04.499030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:04.500974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 106/213] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:04.508346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.516203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:04.519197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:04.519933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:04.520206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:04.520468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:04.533056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:04.539969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:04.540826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:04.541594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 108/214] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:04.547101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.561005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:04.638137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558fc4a-9479-41b5-a04b-f2c4f5420ad8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:04.654054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe8784-bca8-4991-8da8-edadf863978f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:04.677268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:04.677921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:04.678242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:04.678461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:04.679120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:04.680844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:04.691607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:04.692533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:04.692961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:04.693390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:04.694268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:04.694569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:04.696007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:04.704972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 109/215] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:04.711943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.785568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:04.806328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 107/216] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:04 2022] POST /identity/v3/users => generated 297 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:04.832608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c15e4a-2ea3-49e1-99d4-3bf999d7ffad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:04.875401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:04.876428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:04.876881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:04.877469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] PATH_INFO: `/v3/users/04e13743aef1452b89d7243d88b8febd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:04.895100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=04e13743aef1452b89d7243d88b8febd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:04.897098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:04.899105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 110/217] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:04 2022] GET /identity/v3/users/04e13743aef1452b89d7243d88b8febd => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:04.905273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:04.977307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:05.022834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b5f9-4331-415f-b930-13c60b524424 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:05.061751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:05.062296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.062570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.062844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.136930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:05.138412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:05.202657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:05.252840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 108/218] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:05.261108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:05.399993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2a938-92f9-441e-b234-ea837ca66ec6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:05.437885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:05.446836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:05.447561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.447842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.448112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.453687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:05.453687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e27f38-dc5a-4fdb-8fca-191268ad77c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:05.455243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 111/219] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:05 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:05.461975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:05.613266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02e14-7fff-45a8-89ad-cdd992a86f20 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:05.618052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65010dec-3dfb-46a9-a2cb-b34078b15aed None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.618336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65010dec-3dfb-46a9-a2cb-b34078b15aed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.618611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65010dec-3dfb-46a9-a2cb-b34078b15aed None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.621499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 112/220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:05.625730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca807ef-ab6f-46d2-adcf-578c6b1f3207 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.626016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca807ef-ab6f-46d2-adcf-578c6b1f3207 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.626292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca807ef-ab6f-46d2-adcf-578c6b1f3207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.645393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6ca807ef-ab6f-46d2-adcf-578c6b1f3207 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:05.646626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6ca807ef-ab6f-46d2-adcf-578c6b1f3207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:05.665533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:05.666067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.666342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.666611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.668237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=04e13743aef1452b89d7243d88b8febd) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:05.668546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:05.669991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b496e-b0ce-4324-85f0-eebb9c9a9e51 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:05.685739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 109/221] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:05 2022] GET /identity/v3/role_assignments?user.id=04e13743aef1452b89d7243d88b8febd&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:05.747525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 113/222] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:05.767260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30088e3-91b2-4d8c-a57c-caf9fedb4ee4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.767572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30088e3-91b2-4d8c-a57c-caf9fedb4ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.767811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30088e3-91b2-4d8c-a57c-caf9fedb4ee4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.769936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 110/223] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:05.782154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:05.782458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:05.782707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:05.804541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:05.805763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:05.892511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 114/224] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:06.140787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30088e3-91b2-4d8c-a57c-caf9fedb4ee4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.170817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:06.280724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30088e3-91b2-4d8c-a57c-caf9fedb4ee4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.326229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.326872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.327161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.327475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.343008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:06.348125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:06.348987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:06.349779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 111/225] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:06.355178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.376200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.451707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:06.496101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:06.508798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819325e-0891-4a0e-a7d0-7dae320da0f4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.525937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf111-562a-4640-95a7-dc5faaf2a75e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.546377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.546999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.547313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.547586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.548261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:06.548560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:06.550732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:06.560061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 115/226] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:06.565929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.588766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.589490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.589775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.590042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.615359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:06.620246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:06.621905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:06.622875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:06.628120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 112/227] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:06.633833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.679373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634b7c-6cfe-4003-8051-96f7ae1bf7f6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.700869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:06.715419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.716242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.716684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.717096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.718452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:06.720191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:06.722092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 116/228] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:06.727739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.807627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fd08b-5242-418f-8935-31c887e942f3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.833305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c10a0a-3154-49bd-bfa3-80bc2ba22de4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:06.869033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.869283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:06.869828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.870111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.870386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.870600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:06.871071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:06.871266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:06.871400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:06.871804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:06.872785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:06.873258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:06.874984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:06.876885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 117/229] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:06.883540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.896555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 113/230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:06.904011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:06.919410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:07.010128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:07.071744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c41c-4693-42b9-8c45-164f1a43d43a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:07.086060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c62cee-461e-4af4-8476-f6a8bf2f5c83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:07.123571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:07.124236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.124509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.124782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/32773fad843a4779ba2b5d258e24a241/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.128064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, user_id=32773fad843a4779ba2b5d258e24a241, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:07.129521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:07.141439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:07.142076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.142349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.142666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.144050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:07.145617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:07.147272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 114/231] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:06 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:07.152693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:07.203802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 118/232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:06 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/32773fad843a4779ba2b5d258e24a241/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:07.246122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:07.272869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155486b-b600-4b62-b36f-e5623c9180d9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:07.317910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:07.318575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.318848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.319117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.320491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:07.322029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b8d3d-777f-4375-a493-0e97d59f3b5c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:07.324246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 115/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:07 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:07.330052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:07.400679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f6f4-21a6-4f9e-89d2-2397df032b72 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.401003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f6f4-21a6-4f9e-89d2-2397df032b72 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.401209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31f6f4-21a6-4f9e-89d2-2397df032b72 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.402561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 116/234] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:07.408081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f132-06bd-4a64-8e54-d24ccfccb031 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.408355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f132-06bd-4a64-8e54-d24ccfccb031 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.408725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8f132-06bd-4a64-8e54-d24ccfccb031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.441650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4fd8f132-06bd-4a64-8e54-d24ccfccb031 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:07.443022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4fd8f132-06bd-4a64-8e54-d24ccfccb031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:07.461878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b00bd4-7d54-454f-9796-b7417eceb52a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:07.477374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:07.508649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:07.509669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.510147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.510620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/32773fad843a4779ba2b5d258e24a241/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.513240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, user_id=32773fad843a4779ba2b5d258e24a241, role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:07.515206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294f7d7-0549-4e5c-8653-7c5628b8d297 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:07.524809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 117/235] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:07.625296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 119/236] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:07 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/32773fad843a4779ba2b5d258e24a241/roles/8488c378257a4f629184ae9773b882ac => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:07.664087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7b80-f132-4cb7-bc3c-c28cdd55b77b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.664363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7b80-f132-4cb7-bc3c-c28cdd55b77b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.664892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ea7b80-f132-4cb7-bc3c-c28cdd55b77b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.666419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 118/237] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:07.672609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a351a2-6314-409b-971d-1ea6472dff2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:07.672940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a351a2-6314-409b-971d-1ea6472dff2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:07.673166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a351a2-6314-409b-971d-1ea6472dff2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:07.694944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-68a351a2-6314-409b-971d-1ea6472dff2b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:07.696304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-68a351a2-6314-409b-971d-1ea6472dff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:07.793600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 120/238] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:07.893032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7b80-f132-4cb7-bc3c-c28cdd55b77b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:07.977219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.013958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ea7b80-f132-4cb7-bc3c-c28cdd55b77b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.036649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2752e-738b-4aeb-8395-7da076a42414 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.036931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2752e-738b-4aeb-8395-7da076a42414 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.037207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2752e-738b-4aeb-8395-7da076a42414 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.038689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 121/239] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:08.053249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.053516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.054184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.073663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.073663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.073663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.073663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.074827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:08.076506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:08.078309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 119/240] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:08.084778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.089325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:08.090505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:08.175209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 122/241] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:08.175209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.226547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.251171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d43cd-11c7-4c13-81d3-11b4f396cada None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.312561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.313110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.313392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.313665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.314332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:08.316531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:08.327247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f0133-e9cf-4f17-990a-63c0a8d5b9cd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.344515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.364114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.364850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.365012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.365284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.376964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:08.383707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:08.384601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:08.385386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 123/242] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:08.391069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.474932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 120/243] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:08 2022] POST /identity/v3/users => generated 293 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:08.503477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.513567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.515512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9685ad9-77f1-424b-a5d5-f57dd56e3d98 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.601782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.602495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.602495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.602676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.603312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:08.603616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:08.605048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:08.630728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 124/244] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:08.644297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.646298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e03f-819f-4e40-860c-b1643445ada8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.685404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.687537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.687537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.687537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.702323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:08.708850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:08.709750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:08.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:08.717709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 121/245] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:08.719106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.779544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.792133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bdf435-6ce8-4a62-926b-7de85d7575b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.834753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0d3b7-b199-4d1d-8ac8-515c48a73694 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:08.844736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.845680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.846187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.846657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] PATH_INFO: `/v3/users/be2ffba95b8042c7a4541281e455bd9a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.862729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] RBAC: Authorizing `identity:get_user(user_id=be2ffba95b8042c7a4541281e455bd9a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:08.864550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:08.866445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 125/246] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/users/be2ffba95b8042c7a4541281e455bd9a => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:08.872620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.887890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:08.888517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:08.889187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:08.889478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:08.890159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:08.890461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:08.892172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:08.894289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:08.906334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 122/247] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:08.912116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:08.988880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705274b-dac9-4e44-aa8c-e8188a4380e2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.025406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c8e8f-a2c7-488e-84e7-f8c6a95a05ee None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.025824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.026462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.026741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.027016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.047987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:09.073919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.075933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.075933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.075933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] PATH_INFO: `/v3/users/04e13743aef1452b89d7243d88b8febd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.076310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=04e13743aef1452b89d7243d88b8febd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.077796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.081215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 123/248] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/users/04e13743aef1452b89d7243d88b8febd => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:09.088129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:09.117686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.119218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.138397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:09.219806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485b2c0e-d3d4-4d05-9db6-c06845650ec1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.239403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 126/249] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:09.241596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:09.272436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.273166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.273553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.273823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.274961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.276535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.278592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 124/250] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:09 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:09.284842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:09.350102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:09.363901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071334d6-eb46-41bf-ba77-c96b933684f9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.389925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:09.408913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1ca7e-6164-4ce4-88f6-ca0290738497 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.414248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.414898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.415207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.415486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.417201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.418179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.421061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 127/251] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:09 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:09.427522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:09.453295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.454330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.454813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.455978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/04e13743aef1452b89d7243d88b8febd/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.458175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, user_id=04e13743aef1452b89d7243d88b8febd, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.459865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef82822-c189-4be8-b77c-35fa913b3d66 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.540211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 125/252] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:09 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/04e13743aef1452b89d7243d88b8febd/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:09.608414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb234111-e5c3-49d7-a544-8fe0a3531766 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:09.657387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:09.657915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.658191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.658509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.659315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=be2ffba95b8042c7a4541281e455bd9a) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:09.659621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:09.661072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f7d0b-e8b3-4b15-83b9-cac6aa453528 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:09.672796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 128/253] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:09 2022] GET /identity/v3/role_assignments?user.id=be2ffba95b8042c7a4541281e455bd9a&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=fc5855d774c84359acf01fccb8000531 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:09.782332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e0fb-a682-449a-94a8-61f4a1dd2ac0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.782962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e0fb-a682-449a-94a8-61f4a1dd2ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.783515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2e0fb-a682-449a-94a8-61f4a1dd2ac0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.785276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 126/254] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:09.791201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1f75-41a9-4ee0-ac09-08c04f6c6ce3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:09.791492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1f75-41a9-4ee0-ac09-08c04f6c6ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:09.791935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf1f75-41a9-4ee0-ac09-08c04f6c6ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:09.823573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2fbf1f75-41a9-4ee0-ac09-08c04f6c6ce3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:09.825050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2fbf1f75-41a9-4ee0-ac09-08c04f6c6ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:09.895695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:09.926064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 129/255] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:10.130332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b689554-c8c6-47ca-a588-9bb88aadb124 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:10.130696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b689554-c8c6-47ca-a588-9bb88aadb124 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:10.130914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b689554-c8c6-47ca-a588-9bb88aadb124 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:10.132552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 127/256] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:10.137905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:10.138192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:10.138468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:10.168602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:10.170412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:10.301051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 130/257] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:10.320172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b689554-c8c6-47ca-a588-9bb88aadb124 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:10.465795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b689554-c8c6-47ca-a588-9bb88aadb124 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:10.488433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:10.513906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:10.514813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:10.515315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:10.515790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:10.533938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:10.539129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:10.546777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:10.547615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 128/258] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:10.549575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:10.676964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28809c61-e05a-43dc-8f08-dcb8ea28abbb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:10.690272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:10.719851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:10.720745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:10.721239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:10.721746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:10.724025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:10.724484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:10.726039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:10.734671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 131/259] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:10.740459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:10.866720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6ec22-52cc-40d6-ae69-d00ca7e63339 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:10.894228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:10.904969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:10.905786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:10.906240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:10.906656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:10.908029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:10.910353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:10.912119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 129/260] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:10 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:10.929998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:10.996634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:11.092954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cc302-28ce-4fb6-8133-5539d5c4a808 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.107066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69e1d3-7e2d-4166-87ac-684a635e1ba3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.154351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.155379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.155844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.156393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.172440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.173173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.173523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.173899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.192307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:11.198477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:11.199419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:11.200219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:11.210607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 132/261] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:11.217499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:11.240029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:11.241845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:11.286705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:11.291627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:11.352228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 130/262] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:14:11.358151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:11.465203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8783b0c-301d-4a84-bc09-df1a5e30d21d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.468645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2349d-8e9c-4e56-aed3-19a63a8116ef None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.528169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866837-a747-4ada-b393-c5cae638120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.529005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866837-a747-4ada-b393-c5cae638120c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.529383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866837-a747-4ada-b393-c5cae638120c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.529848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866837-a747-4ada-b393-c5cae638120c None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.531207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36866837-a747-4ada-b393-c5cae638120c None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:11.532531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.532919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36866837-a747-4ada-b393-c5cae638120c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:11.533616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.534043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.534452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.535176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 131/263] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:11 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:11.535464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:11.535888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:11.537397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:11.547174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866837-a747-4ada-b393-c5cae638120c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:11.553681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 133/264] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:11.558757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:11.562986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:11.731959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866837-a747-4ada-b393-c5cae638120c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.751786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:11.771144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7f570-18e4-4efd-a5b0-fa5b2ebd7fb9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:11.774909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.775674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.776053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.776429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.777272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=32773fad843a4779ba2b5d258e24a241) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:11.777683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:11.779282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:11.786328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 132/265] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:11 2022] GET /identity/v3/role_assignments?user.id=32773fad843a4779ba2b5d258e24a241&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:11.849206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:11.849872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:11.850147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:11.850413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] PATH_INFO: `/v3/users/32773fad843a4779ba2b5d258e24a241` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:11.851615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] RBAC: Authorizing `identity:get_user(user_id=32773fad843a4779ba2b5d258e24a241)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:11.853093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ac2c1-a3d0-4845-b96f-df6d40aa5180 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:11.855749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 134/266] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:11 2022] GET /identity/v3/users/32773fad843a4779ba2b5d258e24a241 => generated 283 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:11.860571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:11.987278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.024255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29987d1-1833-4ca7-a11f-0cca89cb8429 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:12.040250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79e8-d0c5-494b-9695-a32e55c5eb53 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.040547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79e8-d0c5-494b-9695-a32e55c5eb53 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.040803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a79e8-d0c5-494b-9695-a32e55c5eb53 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.042259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 135/267] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:12.047470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc17fd-859c-48fb-9e54-87cac7ba0031 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.047749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc17fd-859c-48fb-9e54-87cac7ba0031 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.048246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc17fd-859c-48fb-9e54-87cac7ba0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.070220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:12.071309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.071707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.072045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.093477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-77fc17fd-859c-48fb-9e54-87cac7ba0031 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:12.094964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77fc17fd-859c-48fb-9e54-87cac7ba0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:12.169287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:12.170744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:12.183337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 136/268] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:12.185346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2845b1-7f7a-4dbe-a248-e0728bbaea4b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.185840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2845b1-7f7a-4dbe-a248-e0728bbaea4b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.186729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2845b1-7f7a-4dbe-a248-e0728bbaea4b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.188499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 137/269] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:12.196836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.197344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.197826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.234482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:12.237035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:12.292232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.344616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 133/270] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:12.350505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.357240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 138/271] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:12.475731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f174e7d-b425-4890-8d0f-6cfb61fe83ea None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:12.497752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.512967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:12.513949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.514440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.514912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.516491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:12.518802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:12.520964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 134/272] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:12 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:12.527017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.536597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.576779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.640448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88347e-5e86-4f6c-903b-04b9db2fa214 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:12.651301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d92e5-a864-41ab-abd3-41bb53456ccc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:12.678109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:12.679068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.679577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.680046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.680969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=32773fad843a4779ba2b5d258e24a241) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:12.681498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:12.683164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c6046-fcf5-42e0-a430-ccc9a597ad62 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:12.690022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 135/273] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:12 2022] GET /identity/v3/role_assignments?user.id=32773fad843a4779ba2b5d258e24a241&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=8488c378257a4f629184ae9773b882ac => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:12.696353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e31e21-4b75-493a-91bb-ee6e9b5d0b47 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.696826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e31e21-4b75-493a-91bb-ee6e9b5d0b47 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.697253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e31e21-4b75-493a-91bb-ee6e9b5d0b47 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.705770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 136/274] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:12 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:12.707992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:12.708665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.708943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.709211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.709317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.709707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.710162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.736017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:12.741512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:12.742776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:12.743856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:12.744917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:12.745651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:12.747555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 139/275] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:12.750251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.837883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 137/276] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:12.839268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.856413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.882563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9656ec-1f8a-496e-a6d5-d5abc88c64b7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:12.890607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:12.922366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:12.923001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:12.923334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:12.923601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:12.927909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:12.930291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:12.932823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 140/277] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:12.938285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:12.984220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91d0b4-7ef5-48d7-9d9f-24ed7e6848f0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.036066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.037213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.038243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.038532 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.040670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:13.040670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.040939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4028ee-8895-4a4b-9d57-52c89dd3cb5e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.052574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 138/278] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:13.053940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93cc0b-ee7c-4528-ba77-809a825ac359 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.054230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93cc0b-ee7c-4528-ba77-809a825ac359 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.054498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93cc0b-ee7c-4528-ba77-809a825ac359 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.055742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 139/279] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:13.062751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93cc0b-ee7c-4528-ba77-809a825ac359 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.072523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1217b-6b59-4d7a-ab56-f46fb577009d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.112431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.113043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.113321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.113594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.114356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.115947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.162488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.167035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93cc0b-ee7c-4528-ba77-809a825ac359 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.200204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.200828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.201105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.201377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] PATH_INFO: `/v3/users/b9d3e462492042cd96e988d9b7f25762` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.216008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] RBAC: Authorizing `identity:get_user(user_id=b9d3e462492042cd96e988d9b7f25762)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.217450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee8434-cdbe-4482-b036-684b285813bb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.219049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 140/280] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/users/b9d3e462492042cd96e988d9b7f25762 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:13.224691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.224691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.224691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.239164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:13.240356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:13.245964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 141/281] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:12 2022] POST /identity/v3/users => generated 293 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:14:13.246749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.253872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.327284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 141/282] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:13.327284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.384941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708dc4f-f91a-4c64-993d-ed84884ad0b6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.429124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.430542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.431289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.431763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.446659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:13.453195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:13.454101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:13.455469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:13.456071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 142/283] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:13.462442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.465664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.486781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb091c4-57c0-4e54-b9b0-c7575e29e13a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.537566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.538719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.538719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.538719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.541573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, user.id=b9d3e462492042cd96e988d9b7f25762, scope.system=all) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:13.541573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.541781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.542431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.549383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 142/284] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/role_assignments?user.id=b9d3e462492042cd96e988d9b7f25762&scope.system=all&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:13.595022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c14b54-a447-4dc7-accd-9e866a4f3472 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.602361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.635927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.636345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.636467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.636793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.637467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:13.638043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.639572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.650964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 143/285] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:13.661066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.749733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0b3d4-ef38-4417-aed8-40624c3475a0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.793422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fe90c-a64d-4805-9fca-73f2f5983ed2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.799902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.800634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.800982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.801573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.815949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:13.820450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:13.824559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:13.825652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:13.826631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:13.828572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 143/286] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:13.834378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.847494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.848390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.848838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.849256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] PATH_INFO: `/v3/users/be2ffba95b8042c7a4541281e455bd9a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.852391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] RBAC: Authorizing `identity:get_user(user_id=be2ffba95b8042c7a4541281e455bd9a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.854315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:13.856092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 144/287] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/users/be2ffba95b8042c7a4541281e455bd9a => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:13.860370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:13.872222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:13.957273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f7772-e83d-46f8-a66e-684bf7e997a7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:13.993863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:13.994577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:13.994854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:13.995123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:13.996003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:13.996303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:13.997743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.011281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 144/288] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:14.012593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:14.103656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:14.121503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb3491-8411-443e-9cd1-708493d3fb61 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.122210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f05d6f-1a8b-4ca6-b736-9dfcb80ff36f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.160681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.161348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.161624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.161939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] PATH_INFO: `/v3/users/04e13743aef1452b89d7243d88b8febd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.163314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] RBAC: Authorizing `identity:get_user(user_id=04e13743aef1452b89d7243d88b8febd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.164823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.171245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 145/289] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:14 2022] GET /identity/v3/users/04e13743aef1452b89d7243d88b8febd => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:14.171909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.172564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.172841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.173108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.174629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:14.175610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.177326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.179134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 145/290] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:13 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:14.188723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:14.296102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa06466-3c23-4186-9d49-a7e42442b108 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.333094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:14.338757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723074f7-e869-40e1-8841-55502427b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.339711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723074f7-e869-40e1-8841-55502427b133 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.340721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723074f7-e869-40e1-8841-55502427b133 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.341185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723074f7-e869-40e1-8841-55502427b133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.365975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:14.429408 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234d4df-e5fd-487a-9981-a28e981a57a0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.437207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723074f7-e869-40e1-8841-55502427b133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.438997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723074f7-e869-40e1-8841-55502427b133 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.492333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.494837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.496983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.497456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/be2ffba95b8042c7a4541281e455bd9a/roles/fc5855d774c84359acf01fccb8000531` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.500377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, user_id=be2ffba95b8042c7a4541281e455bd9a, role_id=fc5855d774c84359acf01fccb8000531)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.504455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.576878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 146/291] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:14.580125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723074f7-e869-40e1-8841-55502427b133 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:14.596607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 146/292] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:14 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/users/be2ffba95b8042c7a4541281e455bd9a/roles/fc5855d774c84359acf01fccb8000531 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:14.653746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:14.730738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723074f7-e869-40e1-8841-55502427b133 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.786226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.786874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.787168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.787443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.788579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.790010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07774e87-ce8c-4b96-befa-8ea0d84b523c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.791982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 147/293] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:14 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:14.798306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:14.915252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774476-ed36-480c-a8bf-621028df9bcc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:14.942367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:14.957376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:14.958063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:14.958343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:14.958613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:14.959404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=04e13743aef1452b89d7243d88b8febd) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:14.959713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:14.961152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d50cd53-3d8d-41c4-b1fe-005fc7f27443 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:14.972529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 147/294] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:14 2022] GET /identity/v3/role_assignments?user.id=04e13743aef1452b89d7243d88b8febd&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:15.707370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409af4-e5ff-4edc-ab73-cb8c3b3157b1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.707902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409af4-e5ff-4edc-ab73-cb8c3b3157b1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.707902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c409af4-e5ff-4edc-ab73-cb8c3b3157b1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.709475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 148/295] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:15.714667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e3f8-d262-4510-9fa9-9499468d0a23 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.714950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e3f8-d262-4510-9fa9-9499468d0a23 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.715255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0788e3f8-d262-4510-9fa9-9499468d0a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.734164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0144-610f-4d17-949e-c6743e8f75d4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.734441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0144-610f-4d17-949e-c6743e8f75d4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.734713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae0144-610f-4d17-949e-c6743e8f75d4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.736350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 149/296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:15.737334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0788e3f8-d262-4510-9fa9-9499468d0a23 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:15.738541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0788e3f8-d262-4510-9fa9-9499468d0a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:15.744120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7a3c-3850-448a-b9b0-c181f53c5ef4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.744639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7a3c-3850-448a-b9b0-c181f53c5ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.745110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5a7a3c-3850-448a-b9b0-c181f53c5ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.764246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8f5a7a3c-3850-448a-b9b0-c181f53c5ef4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:15.765839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8f5a7a3c-3850-448a-b9b0-c181f53c5ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:15.827184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 148/297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:15.844661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 150/298] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:15.930324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eead0c-7e0e-4b40-9598-403bf9dbec63 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.930822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eead0c-7e0e-4b40-9598-403bf9dbec63 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.930896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eead0c-7e0e-4b40-9598-403bf9dbec63 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.932500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 149/299] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:15.937714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726053-5450-487d-9d0b-0583eb248262 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:15.938000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726053-5450-487d-9d0b-0583eb248262 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:15.938274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46726053-5450-487d-9d0b-0583eb248262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:15.958153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-46726053-5450-487d-9d0b-0583eb248262 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:15.959282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-46726053-5450-487d-9d0b-0583eb248262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:15.990308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.035362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 151/300] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:16.099763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eead0c-7e0e-4b40-9598-403bf9dbec63 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.130024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46726053-5450-487d-9d0b-0583eb248262 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.217625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eead0c-7e0e-4b40-9598-403bf9dbec63 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.237049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.243456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46726053-5450-487d-9d0b-0583eb248262 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.269636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.270523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.270800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.271119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.272267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.273784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.275467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 150/301] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:16.281209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.290297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.291128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.291602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.292548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.307830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:16.314933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:16.316030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:16.317023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:16.319456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 152/302] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:16.326124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.357428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.396593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b616319-aba5-47ce-8647-78820f78de8e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.433535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.434148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.434424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.434698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] PATH_INFO: `/v3/groups` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.435410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.436867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.438348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94cee-32be-495f-b2a8-1b8ff8ca1637 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.473840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.474364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.474641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.474906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.475601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:16.475912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.477250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.485279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 153/303] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:16.486164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.511450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 151/304] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 1 16:14:16 2022] POST /identity/v3/groups => generated 229 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:16.521144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.589665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.600469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.612596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5753537c-aca8-4c57-a274-63a447651e8c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.654908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.656396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.657231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.657852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.670906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9984e7a-2813-4e91-9619-80ef1444f1fc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.675352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:16.682526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:16.684829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:16.685604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:16.687239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 154/305] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:16.693941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.719360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.720368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.720804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.721257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] PATH_INFO: `/v3/users/84e0aaad9d794486bf5c2b5822d72a5b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.739751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] RBAC: Authorizing `identity:get_user(user_id=84e0aaad9d794486bf5c2b5822d72a5b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.741212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.743015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 152/306] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/users/84e0aaad9d794486bf5c2b5822d72a5b => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:16.749005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.812945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c069-e545-48b4-9a49-d2730ba60d0f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.850285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.850888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.851211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.851484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.852148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:16.852444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.853872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.863444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 155/307] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:16.868795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:16.880709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.883690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c91b5-1eb8-483a-8c4a-728b3274576d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:16.886980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:16.928011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:16.928628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.928907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.929179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.929915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, user.id=84e0aaad9d794486bf5c2b5822d72a5b, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:16.930222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:16.931780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff544e2e-f373-4aa0-b598-34d55d1bdbd2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:16.936992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 153/308] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/role_assignments?user.id=84e0aaad9d794486bf5c2b5822d72a5b&scope.system=all&role.id=fc5855d774c84359acf01fccb8000531 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:16.936992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a1d7-a02b-40e8-a921-d028d7ee4dff None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.936992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a1d7-a02b-40e8-a921-d028d7ee4dff None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.936992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a1d7-a02b-40e8-a921-d028d7ee4dff None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.940588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 154/309] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:16.946522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:16.946857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:16.947067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:16.968891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:16.969861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:16.992334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c08d1-9de1-48bf-b1bd-0e61a4721633 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.037585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.038601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.039073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.039636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] PATH_INFO: `/v3/users/b9d3e462492042cd96e988d9b7f25762` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:17.041335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] RBAC: Authorizing `identity:get_user(user_id=b9d3e462492042cd96e988d9b7f25762)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:17.043255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:17.045209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 156/310] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:16 2022] GET /identity/v3/users/b9d3e462492042cd96e988d9b7f25762 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:17.051933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:17.052727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 155/311] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:17.143225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:17.162873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c2ce7-9a7d-4fac-9389-67074f0fa96e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.200492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.201474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.201946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.202409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] PATH_INFO: `/v3/system/users/b9d3e462492042cd96e988d9b7f25762/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:17.204150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b9d3e462492042cd96e988d9b7f25762, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:17.206121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:17.301382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 157/312] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 1 16:14:17 2022] PUT /identity/v3/system/users/b9d3e462492042cd96e988d9b7f25762/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:17.335028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:17.441902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:17.449975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ccc7b-81fb-4ce8-86a5-d26cf9b0e286 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.488962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.489954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.490455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.490920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:17.502207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:17.507538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:17.508427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:17.509196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:17.511504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 156/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:17.517605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:17.628173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e42096-f97c-44e0-b727-bda0956ce39e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.664639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.665250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.665527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.665800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:17.666491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:17.666774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:17.668198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:17.677525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 158/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:17.683116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:17.789195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6697667-0a66-41dc-87cd-1bbd4189e11a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.825321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.825959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.826236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.826510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] PATH_INFO: `/v3/users/be2ffba95b8042c7a4541281e455bd9a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:17.827680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] RBAC: Authorizing `identity:get_user(user_id=be2ffba95b8042c7a4541281e455bd9a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:17.831456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:17.833227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 157/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:17 2022] GET /identity/v3/users/be2ffba95b8042c7a4541281e455bd9a => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:17.838709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:17.849024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:17.942047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951428f9-7816-4298-8b27-8bec97516dd3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:17.977545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:17.978161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:17.978438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:17.978711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.050153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:18.051909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:18.063711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:18.157554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 159/316] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:18.163787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:18.190416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:18.275538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f4412c-84b7-4f07-8ca7-92f9219307e0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:18.309851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:18.310407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.310691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.310962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.312113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:18.313556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22e31f-ab65-40ad-9db8-d56cb45e3cc3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:18.316055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 158/317] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:18 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:18.321615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:18.426873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9416c9-68a0-465d-97d8-ad7950063190 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:18.434651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:18.464572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:18.465199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.465478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.465748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.466545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3, user.id=be2ffba95b8042c7a4541281e455bd9a) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:18.466856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:18.468317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472be7aa-abef-48e6-93bc-bb088662fa0c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:18.479555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 160/318] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:18 2022] GET /identity/v3/role_assignments?user.id=be2ffba95b8042c7a4541281e455bd9a&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=fc5855d774c84359acf01fccb8000531 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:18.575681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848997c-0e89-4212-83f9-286ddcfd5aac None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.575863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848997c-0e89-4212-83f9-286ddcfd5aac None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.576129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848997c-0e89-4212-83f9-286ddcfd5aac None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.577789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 159/319] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:18.584767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5781-929e-4ac9-852b-5d10c56a6fb5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.585055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5781-929e-4ac9-852b-5d10c56a6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.586648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce5781-929e-4ac9-852b-5d10c56a6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.605641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f3ce5781-929e-4ac9-852b-5d10c56a6fb5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:18.606866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3ce5781-929e-4ac9-852b-5d10c56a6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:18.682127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 161/320] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:18.952310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848997c-0e89-4212-83f9-286ddcfd5aac None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:18.977653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1605d7-aa16-42cc-a9ab-b56e75f5316b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.978158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1605d7-aa16-42cc-a9ab-b56e75f5316b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.978230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1605d7-aa16-42cc-a9ab-b56e75f5316b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:18.979565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 162/321] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:18.984477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:18.984835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:18.985176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.004243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:19.005412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:19.052756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:19.065107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848997c-0e89-4212-83f9-286ddcfd5aac None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.078950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 163/322] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:19.105016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.105658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.105933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.106204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.107332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:19.108837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766de56b-ec8e-49c7-8e21-b129cefce467 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:19.111186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 160/323] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:19.116201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.149221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:19.218865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013cbc-cc1c-44fd-b424-66f535988e52 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.255563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.256174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.256450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.256720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] PATH_INFO: `/v3/groups` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.257352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:19.258790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:19.303667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de947d-11bc-4071-97b7-62c70d258b8d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.304174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de947d-11bc-4071-97b7-62c70d258b8d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.304256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de947d-11bc-4071-97b7-62c70d258b8d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.305511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 161/324] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:19.310237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.310515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.310786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.329375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:19.330551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:19.368785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 164/325] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Sep 1 16:14:19 2022] POST /identity/v3/groups => generated 226 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:19.368785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.407478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 162/326] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:19.477492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:19.488392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456f602-15c9-4996-b919-b983d2e48402 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.527545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.528177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.528457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.528779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.539786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:19.544948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:19.545818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:19.546579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:19.551552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 165/327] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:19.553100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.686181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.701320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:19.716929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce41713-aec9-4ff1-89ab-2c2dbfcf176a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.761144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.761686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.761967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.762242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.762917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:19.769182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:19.769182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:19.776716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 163/328] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:19.782070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.819809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a47ee-306d-4c00-99af-4828b69c53fa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.864695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.865342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.865623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.865896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.877391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:19.882590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:19.883522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:19.884308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 166/329] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:19.891401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:19.906757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:19.919419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05265131-8d81-4002-852a-cdf721c31c32 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:19.968767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:19.969990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:19.970997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:19.972072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] PATH_INFO: `/v3/users/84e0aaad9d794486bf5c2b5822d72a5b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:19.973721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] RBAC: Authorizing `identity:get_user(user_id=84e0aaad9d794486bf5c2b5822d72a5b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:19.975438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:19.977750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 164/330] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:19 2022] GET /identity/v3/users/84e0aaad9d794486bf5c2b5822d72a5b => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:19.983231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:20.009545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:20.022564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea06e-fc94-40d8-9447-0065dce065fa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:20.056935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:20.057715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:20.058096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:20.058469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:20.059274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:20.059756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:20.061251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:20.071243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 167/331] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:20.085259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:20.107548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09502483-843f-484f-a38f-52dab4b5cb4c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:20.150844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:20.151631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:20.151914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:20.152293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] PATH_INFO: `/v3/system/users/84e0aaad9d794486bf5c2b5822d72a5b/roles/fc5855d774c84359acf01fccb8000531` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:20.153808 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=84e0aaad9d794486bf5c2b5822d72a5b, role_id=fc5855d774c84359acf01fccb8000531)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:20.155499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:20.178453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:20.213377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77116298-6279-43ad-97ae-036b74d2d5d8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:20.238379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 165/332] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Sep 1 16:14:19 2022] PUT /identity/v3/system/users/84e0aaad9d794486bf5c2b5822d72a5b/roles/fc5855d774c84359acf01fccb8000531 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:20.268515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:20.269176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:20.269442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:20.269715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] PATH_INFO: `/v3/users/b9d3e462492042cd96e988d9b7f25762` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:20.273774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] RBAC: Authorizing `identity:get_user(user_id=b9d3e462492042cd96e988d9b7f25762)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:20.275322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df332999-2764-495e-8621-0ff7e9ad6f3c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:20.277174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 168/333] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:20 2022] GET /identity/v3/users/b9d3e462492042cd96e988d9b7f25762 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:20.284062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:20.409647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:20.433852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3961b5-6848-4737-bbc3-6aeddc9da4e1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:20.478825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:20.479464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:20.479740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:20.480008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:20.480740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] RBAC: Adding query filter params (role.id=184b9cf9fc5144009f8b26213f237b6c, user.id=b9d3e462492042cd96e988d9b7f25762, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:20.481212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:20.482654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92adc9-9bba-4055-825c-5331e32e8b84 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:20.500170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 166/334] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 16:14:20 2022] GET /identity/v3/role_assignments?user.id=b9d3e462492042cd96e988d9b7f25762&scope.system=all&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 510 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:21.685579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820bb6-069f-4b02-8e3b-b56ec822be6d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:21.686124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820bb6-069f-4b02-8e3b-b56ec822be6d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:21.686124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92820bb6-069f-4b02-8e3b-b56ec822be6d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:21.687704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 169/335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:21.694220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee272-2ac6-4427-b034-166939ecf5e0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:21.694681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee272-2ac6-4427-b034-166939ecf5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:21.694753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fee272-2ac6-4427-b034-166939ecf5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:21.721264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-11fee272-2ac6-4427-b034-166939ecf5e0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:21.722794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-11fee272-2ac6-4427-b034-166939ecf5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:21.810463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 167/336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:21.935799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556a63-c5d5-468a-badc-60807015bf57 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:21.936397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556a63-c5d5-468a-badc-60807015bf57 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:21.936887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556a63-c5d5-468a-badc-60807015bf57 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:21.938632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 170/337] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:21.948679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:21.949261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:21.949731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:21.969436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:21.971023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:22.045889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:22.051316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 168/338] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:22.367409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556a63-c5d5-468a-badc-60807015bf57 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:22.493915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556a63-c5d5-468a-badc-60807015bf57 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:22.533936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:22.534569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:22.534843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:22.535113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:22.544677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:22.549875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:22.555116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:22.555994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:22.556809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 171/339] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:22.562416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:22.609755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:22.733573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268331d-3d29-45ba-bc03-a71b5638aebb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:22.779978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:22.780594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:22.780869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:22.781138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:22.781810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:22.782172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:22.783758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:22.787020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e94dc-aaa3-4bac-9ea4-7631075badb5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:22.805431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 169/340] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:22.811471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:22.864006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:22.864665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:22.864942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:22.865210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:22.884720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:22.890734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:22.891692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:22.893044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:22.893632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 172/341] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:22.898986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:22.929671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:22.983525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.010367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6a46c-10ac-4723-8bd8-477f8f31581c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.035755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edaf07-b654-41c9-9df6-bbbfcc96466a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.077603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.078132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.078410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.078681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.079445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:23.079930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.081697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e5fdf-e5a3-4bdb-8eb1-47ac3ba697c3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.090480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 173/342] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:23.091783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b881-62c9-440c-a1cd-c79e2f0ef4eb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.092065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b881-62c9-440c-a1cd-c79e2f0ef4eb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.092336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249b881-62c9-440c-a1cd-c79e2f0ef4eb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.093547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.093747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 174/343] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:23.094169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.094495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.095202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.098728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249b881-62c9-440c-a1cd-c79e2f0ef4eb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.218836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.220529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565e517-b86f-4116-8d37-6300704edc6e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.227734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249b881-62c9-440c-a1cd-c79e2f0ef4eb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.268223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.277453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.278355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.278640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.278911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.292659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.350646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 170/344] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:23.352616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33a274-8aab-42ac-936a-cbaff2a70161 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.353017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33a274-8aab-42ac-936a-cbaff2a70161 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.353296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33a274-8aab-42ac-936a-cbaff2a70161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.366866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.368367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4b365-9a53-4dab-88fd-86b76efdf682 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.377609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ba33a274-8aab-42ac-936a-cbaff2a70161 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:23.379246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ba33a274-8aab-42ac-936a-cbaff2a70161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:23.474639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 171/345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:23.476013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2682-8634-4504-a45d-d06733ae1ae0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.476297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2682-8634-4504-a45d-d06733ae1ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.476565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd2682-8634-4504-a45d-d06733ae1ae0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.478068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 172/346] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:23.478940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd2682-8634-4504-a45d-d06733ae1ae0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.507265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 175/347] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:23.508562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.508845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.509252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.544243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:23.545456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:23.577070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.599113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.610286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd2682-8634-4504-a45d-d06733ae1ae0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.652155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.652948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.653235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.653505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.654637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.656427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.658785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 173/348] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:23.659591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.691037 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 176/349] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:23.692058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.778272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859bacf-08d9-4d83-8e13-9105e183ddf7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.815550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.816196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.816473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.816743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.817877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.819346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.821263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 174/350] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:23.826591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.847955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e644005-7ca0-47d8-8980-c26e1ebc6c3c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.857993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.888358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.888911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.889187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.889447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.899116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.900603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.902526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 177/351] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:23.908993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:23.929591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:23.941960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd644291-32dd-4fd5-b3ed-80725ebb1938 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:23.980831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:23.981472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:23.981743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:23.982011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:23.983126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:23.984637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:23.991603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 175/352] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:23.993517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.024474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae484b28-e461-4f73-878b-bae7d9188db2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.063270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.064632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.065122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.065419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.066214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=bc0910ca971f4c5186644553060c147f) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:24.066523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:24.068058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ea857-8a93-47e7-a15f-c1d26248e4df None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:24.076940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 178/353] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:24.078241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce22fa-b737-4655-adbf-2a1c8d0fd607 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.078522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce22fa-b737-4655-adbf-2a1c8d0fd607 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.078790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ce22fa-b737-4655-adbf-2a1c8d0fd607 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.080770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 179/354] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:24.086478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.086763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.087006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.099687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.106637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb18ee-291a-4b41-994c-88f4b634142e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.116384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:24.117967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:24.144257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.144784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.145063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.145335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.146369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=bc0910ca971f4c5186644553060c147f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:24.146896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:24.149485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:24.158191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 176/355] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:23 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=8488c378257a4f629184ae9773b882ac => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:24.159044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.230077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 180/356] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:24.230857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.255181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.305138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2d528-c196-45f6-a17b-202bf08f9d7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.347486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.348180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.348460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.348746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.361505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5c9281c9-bc5e-4c31-a856-ed05f079fdb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:24.369029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:24.369994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 01 16:14:24.370788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:24.371411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 177/357] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:24.375181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5f80-4d4d-4101-882b-668df0d6b055 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.375181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5f80-4d4d-4101-882b-668df0d6b055 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.375181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e5f80-4d4d-4101-882b-668df0d6b055 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.375181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 178/358] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:24.379275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5f80-4d4d-4101-882b-668df0d6b055 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.382875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c71e4b-a376-44c8-8a32-32f898d84ee2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.412231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.427457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.428421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.428837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.429245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.443422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-14e0fda6-8f85-43f1-a2ea-68e07bb8a8de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:24.450457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:24.451668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:24.452983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:24.455708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 181/359] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:24.457242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb732922-f595-444c-985c-d2c569dee285 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.457660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb732922-f595-444c-985c-d2c569dee285 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.458073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb732922-f595-444c-985c-d2c569dee285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.482009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cb732922-f595-444c-985c-d2c569dee285 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:24.483806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cb732922-f595-444c-985c-d2c569dee285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:24.511834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e5f80-4d4d-4101-882b-668df0d6b055 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.552455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.553226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.553603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.553972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.554746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:24.555140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:24.557462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:24.568042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.570080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 179/360] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:24.571011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.572453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 182/361] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:24.576899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb732922-f595-444c-985c-d2c569dee285 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.730555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.744050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb732922-f595-444c-985c-d2c569dee285 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.764188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513a324-32da-4b8c-adb0-81241e3d6445 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:24.795144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.796100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.797517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.798572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] PATH_INFO: `/v3/users/84e0aaad9d794486bf5c2b5822d72a5b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.802129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] RBAC: Authorizing `identity:get_user(user_id=84e0aaad9d794486bf5c2b5822d72a5b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:24.808402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:24.810257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 183/362] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/users/84e0aaad9d794486bf5c2b5822d72a5b => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:24.811282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.854495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:24.856319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:24.856966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:24.859294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:24.860154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:24.861435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:24.863639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:24.881748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 180/363] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:24.884673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:24.940134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:24.991505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5f8bf-d8fe-42d0-82d6-0d6baaba461c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.009262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5b552-5607-4e6c-bdc5-edcb60d7e554 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.029472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.030117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.030396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.030675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.043775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:25.050904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.052305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.052769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.053258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] RBAC: Adding query filter params (role.id=fc5855d774c84359acf01fccb8000531, user.id=84e0aaad9d794486bf5c2b5822d72a5b, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:25.055042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.056753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:25.057113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:25.057968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:25.058759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:25.063052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 184/364] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:25.064099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.070162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 181/365] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Sep 1 16:14:24 2022] GET /identity/v3/role_assignments?user.id=84e0aaad9d794486bf5c2b5822d72a5b&scope.system=all&role.id=fc5855d774c84359acf01fccb8000531 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:25.071123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.074461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.203184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914c59ea-0051-4dfb-b93a-17a807534483 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.244688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.245704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.245995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.246274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.259560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:25.265870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:25.266787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:25.267778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 182/366] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:25.268440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.281153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2a14b-7f71-454e-9a90-2ee221bae290 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.319108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.331244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.338504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.338504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.338504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.373229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.404958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e1d1b-c013-4ae8-999a-6577389d6fbd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.434667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.436249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.458058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.458672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.459280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.459557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.460234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:25.460536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.462442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.476334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 183/367] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:25.477479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.564194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 185/368] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:25.565162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.605108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.615779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f2cc1-5cfa-405e-8602-cb9f36cb35e3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.647377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.655880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.656444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.656727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.657008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.659082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:25.659537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.660973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.673252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 184/369] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:25.674510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.714395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333bcdd-7dc5-4c15-9ea9-1d90b4137d30 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.761891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.762514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.762793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.763064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.798949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc1a9b-29b4-4505-98cb-84bdbb8a5c7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.840856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:25.841440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:25.841717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:25.841989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:25.843190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.844711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.847924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 185/370] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:25.848873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:25.855074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:25.856561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:25.893803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.921095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:25.983012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a201a5-d781-4f7e-bdf9-b4ded5cdabf4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:25.994522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 186/371] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:25.995524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.035806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.036688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.037345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.038631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.109771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d4dcd-a3e9-4759-93b2-ccaf7e418eb5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.134046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.136042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.141375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.150460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.151162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.151461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.156761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.156761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.156761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.159006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 187/372] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.159885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.211110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.273703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 186/373] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:26.275138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.279860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196710c-a968-4089-bf68-3dc596255765 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.335132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.336343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.336823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.337236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.339288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.341513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.343772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 188/374] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.344677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.407162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c152d5a5-2b99-4a05-add5-cb47c428a679 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.428414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.454060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.455355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.456036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.456455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.457489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:26.457940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.459901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.467910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 187/375] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.469357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.476392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959f6df-bebc-4293-a4a0-d80cda2413fb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.497133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.561971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.562963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.563472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.564490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] PATH_INFO: `/v3/projects/18a3d39073f24e449904bb491fbe8869` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.568803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] RBAC: Authorizing `identity:get_project(project_id=18a3d39073f24e449904bb491fbe8869)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.570794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.580886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 189/376] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/projects/18a3d39073f24e449904bb491fbe8869 => generated 344 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.584419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.604398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41575-8c34-4c78-ad2e-500c1ae4f7d4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.649445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.650706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.651433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.651852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.653159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.655015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.658335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 188/377] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.664594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.769638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.787278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb1fd2-b671-4f74-b866-806895cd9792 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.790909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbb1d3-30e5-4421-8a91-e357328a3501 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:26.824834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:26.830913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.831671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.832046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.832372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] PATH_INFO: `/v3/groups/de380a20db694174bbdaf16f0cc4232d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.836631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:26.837240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.837518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.837790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.838197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] RBAC: Authorizing `identity:get_group(group_id=de380a20db694174bbdaf16f0cc4232d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.838521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:26.838827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:26.839692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b1de6-ff0b-4a3d-94e4-032ad06e846c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.842254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 190/378] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/groups/de380a20db694174bbdaf16f0cc4232d => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:26.842373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f0f99-1e9d-411a-b088-182f88588a1c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:26.843805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e4a15-2d47-4cbe-b11c-ef94e58e0ff2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.844136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e4a15-2d47-4cbe-b11c-ef94e58e0ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.844414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e4a15-2d47-4cbe-b11c-ef94e58e0ff2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.847188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 191/379] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:26.847740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba32c-90c7-4303-a6ba-f678cc07ef3c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.848026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba32c-90c7-4303-a6ba-f678cc07ef3c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.848296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ba32c-90c7-4303-a6ba-f678cc07ef3c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.849920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 192/381] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:26.850061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 189/381] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=8488c378257a4f629184ae9773b882ac => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:26.850748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ba32c-90c7-4303-a6ba-f678cc07ef3c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:26.852142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835de-5871-4c33-884a-9e18889a7c48 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.852599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835de-5871-4c33-884a-9e18889a7c48 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.853074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791835de-5871-4c33-884a-9e18889a7c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.879551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-791835de-5871-4c33-884a-9e18889a7c48 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:26.880842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-791835de-5871-4c33-884a-9e18889a7c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:26.972717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 190/382] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:26.974127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:26.974414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:26.974755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:26.999252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ba32c-90c7-4303-a6ba-f678cc07ef3c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.016331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:27.017903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:27.042072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.043042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.043559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.044078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.045322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] RBAC: Adding query filter params (group.id=de380a20db694174bbdaf16f0cc4232d, role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=18a3d39073f24e449904bb491fbe8869) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:27.045783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:27.047442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:27.054562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 193/383] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:26 2022] GET /identity/v3/role_assignments?group.id=de380a20db694174bbdaf16f0cc4232d&scope.project.id=18a3d39073f24e449904bb491fbe8869&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:27.115683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 191/384] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:27.294557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.358936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:27.403569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.410341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cadb2a-0483-4d87-bc78-1d4123599f6b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.427652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:27.457416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.458558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.459240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.459706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.472892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:27.480734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:27.481619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:27.482401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:27.484979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 194/385] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:27.491315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.558338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d464030-2fab-4a43-9d07-051db4c81c04 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.609221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.609868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.610142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.610442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.622343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:27.627766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:27.628636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:27.629418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 192/386] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:27.637252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.647358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2270d1-c512-410a-bdae-dfaa2ab73086 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.674716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:27.693080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.693688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.693966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.694295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.694974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:27.695322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:27.696694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:27.707280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 195/387] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:27.713435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.728512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:27.769163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72eea4-7062-4b95-95b5-0636e15cc1ee None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.818894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.819455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.819732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.820000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.820668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:27.820966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:27.822346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:27.833259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 193/388] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:27.838991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:27.844501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e34673-6d3f-4196-944b-baa9fce571b9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:27.898589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:27.900782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:27.900782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:27.900782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:27.901432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:27.902891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:27.904901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 196/389] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:27.911998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:28.007182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea8139-e3a6-43db-9b8a-b115f19d97ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:28.020115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:28.058224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:28.064434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:28.065453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.065931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.066434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.067787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:28.069653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:28.072621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 194/390] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:28.078587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:28.110291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a3936-a8ea-4821-94dd-571ca4280509 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:28.202353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc00485-9785-42bf-8c76-81fcc95a948a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:28.203869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:28.204547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.204823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.205092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.206270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:28.207754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:28.212180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 197/391] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:27 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:28.215411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:28.269107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:28.270385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.270868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.271606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.273977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:28.276656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:28.284003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 195/392] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:28 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:28.286768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:28.410201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:28.443890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574db17-87be-452b-afdc-146a613eb1ae None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:28.477032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87268148-40e5-4873-9142-8fe8b1cfc6f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:28.484123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:28.503608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:28.504281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.504615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.517302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.517302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, group_id=1a03050e0c884c9b8ab3d2998a47f741, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:28.517302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4aa196-e52e-44d4-a290-dfff7bd7b2ed None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:28.542806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:28.543622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.543906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.544245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.546532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] RBAC: Authorizing `identity:create_grant(project_id=bc0910ca971f4c5186644553060c147f, group_id=1a03050e0c884c9b8ab3d2998a47f741, role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:28.548151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96d9ff-5078-4e95-a26d-3c7dd44a349f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:28.598899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 198/393] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 16:14:28 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:28.633082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 196/394] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 16:14:28 2022] PUT /identity/v3/projects/bc0910ca971f4c5186644553060c147f/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/8488c378257a4f629184ae9773b882ac => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:28.945892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0a099-95e3-4313-ba73-d7cda2d6fa8c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.946641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0a099-95e3-4313-ba73-d7cda2d6fa8c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.947123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0a099-95e3-4313-ba73-d7cda2d6fa8c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.949201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 199/395] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:28.955022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccfc4b-ab44-4c8f-b642-1312a0976144 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:28.955326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccfc4b-ab44-4c8f-b642-1312a0976144 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:28.955843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccfc4b-ab44-4c8f-b642-1312a0976144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:28.982706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dfccfc4b-ab44-4c8f-b642-1312a0976144 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:28.984247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dfccfc4b-ab44-4c8f-b642-1312a0976144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:29.085776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 197/396] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:29.561428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ba8-6da0-453b-8dae-ac3b09b1e369 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:29.562460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ba8-6da0-453b-8dae-ac3b09b1e369 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:29.562952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc71ba8-6da0-453b-8dae-ac3b09b1e369 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:29.565229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 200/397] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:29.574332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:29.574920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:29.575448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:29.600858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:29.603026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:29.611238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc71ba8-6da0-453b-8dae-ac3b09b1e369 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:29.719535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:29.748273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc71ba8-6da0-453b-8dae-ac3b09b1e369 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:29.752752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 198/398] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:29.785889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:29.786523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:29.786798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:29.787069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:29.798696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:29.803774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:29.804685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:29.805535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:29.809652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 201/399] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:29.811033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:29.915034 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c2133-c00f-44e1-b230-437e7f8e4356 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:29.956008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:29.956955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:29.957417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:29.957870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:29.958720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:29.959213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:29.960783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fddadc-45ab-4f5a-9e95-8a3dc48811d4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:29.975023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 199/400] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:29.976678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.022437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.090840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c4b70-b940-4a0c-8bbf-a5dffa432ebe None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.127431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.128249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0af8-77cf-4764-884e-53eb68b383dc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.128689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0af8-77cf-4764-884e-53eb68b383dc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.128861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f0af8-77cf-4764-884e-53eb68b383dc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.129518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.130022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.130463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 200/401] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:30.130724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.139661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:30.141321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa7436-669c-4e9f-813f-cc96af46fd93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:30.143646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 202/402] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:29 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:30.149314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0af8-77cf-4764-884e-53eb68b383dc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.150048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.150513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.150958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.172779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:30.174797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:30.255022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 203/403] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:30.289212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.302866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.343565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.420564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1da155-ba27-46e5-8a49-5de1484ca769 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.459073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.459668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.460091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.460367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.470651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f0af8-77cf-4764-884e-53eb68b383dc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.473036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:30.479584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:30.480470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:30.481252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:30.481857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 204/404] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:30.486912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.573228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.573868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.574304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.574735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.580440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:30.582098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:30.584246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 201/405] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:30.599533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.604619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5156-a5ff-461e-b565-9f0f3104fa72 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.608156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.644947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.645967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.646249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.646570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.647333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:30.647634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:30.649083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:30.660432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 205/406] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:30.661851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.751404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621718-b5a5-4c17-bf64-b1f94c0a13b9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.778048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.797003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b17251-6560-44b6-9af1-9c7ecbeaa7ac None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:30.804641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.804641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.804641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.804641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.804979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, group_id=1a03050e0c884c9b8ab3d2998a47f741, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:30.805635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:30.841421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:30.842052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:30.842330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:30.842601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:30.854201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:30.859513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:30.860400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:30.861238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:30.865698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 206/407] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:30.866544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:30.887723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:30.893798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 202/408] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 16:14:30 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:30.894945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.035462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e996f7-d611-4ee1-b2a8-c7ccb6915699 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.035825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7ce87-493f-4ef9-9bcb-2cd1a498fe55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.073891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.074527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.074890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.075193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.075493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.076102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.076376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.076641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.076955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.077307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:31.078370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.078468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e36aa24-c0ac-40b0-bdce-d023a659afff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.080284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b44de26-875f-49e8-8ff0-a5c4bba0cab8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.084164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 207/409] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:31.084164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db46ec6-f0d6-4491-ace3-846732649635 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.084164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db46ec6-f0d6-4491-ace3-846732649635 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.084164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db46ec6-f0d6-4491-ace3-846732649635 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.084164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 208/410] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:31.084978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d586-82ac-4287-9538-c72624ccfb31 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.085078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d586-82ac-4287-9538-c72624ccfb31 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.085829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4d586-82ac-4287-9538-c72624ccfb31 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.087055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 209/411] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:31.088447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d586-82ac-4287-9538-c72624ccfb31 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.097016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 203/412] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:31.099395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b9d5e-5191-445c-aeea-bcca7e42465b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.099395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b9d5e-5191-445c-aeea-bcca7e42465b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.099395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b9d5e-5191-445c-aeea-bcca7e42465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.118451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-175b9d5e-5191-445c-aeea-bcca7e42465b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:31.119775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-175b9d5e-5191-445c-aeea-bcca7e42465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:31.158468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.180387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.194733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 204/413] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:31.199371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.199371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.199371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.205168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4d586-82ac-4287-9538-c72624ccfb31 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.220200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:31.221782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:31.245364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.245364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.245364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.245364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.247304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.252827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.252827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 210/414] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:31.253463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.309997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 205/415] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:31.311715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.372967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3b52a-9643-445e-a478-b8b645be0447 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.407940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.408842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.408952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.409183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] PATH_INFO: `/v3/groups/de380a20db694174bbdaf16f0cc4232d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.410294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] RBAC: Authorizing `identity:get_group(group_id=de380a20db694174bbdaf16f0cc4232d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.411757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.413681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 211/416] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/groups/de380a20db694174bbdaf16f0cc4232d => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:31.418834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.431736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.435658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f26379-7cd9-44b7-bb0f-3bdce59ddbd8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.455601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.484375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.486016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.486605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.487161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.489765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] RBAC: Authorizing `identity:create_grant(project_id=cce8cfcae70e4b97b428d49b21a7acd3, group_id=1a03050e0c884c9b8ab3d2998a47f741, role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.491498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.524486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703321e-7660-48b0-be6e-fb7661b8d179 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.548101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 206/417] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 16:14:31 2022] PUT /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3/groups/1a03050e0c884c9b8ab3d2998a47f741/roles/8488c378257a4f629184ae9773b882ac => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:31.550190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.574682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.574682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.574682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.575025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] PATH_INFO: `/v3/projects/18a3d39073f24e449904bb491fbe8869` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.576531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] RBAC: Authorizing `identity:get_project(project_id=18a3d39073f24e449904bb491fbe8869)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.578163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.581149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 212/418] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/projects/18a3d39073f24e449904bb491fbe8869 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:31.592386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.694246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53481a69-4457-47a7-9eec-bfa17e706b33 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.738044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.738688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.739615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.739900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.742944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.755805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:31.756472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7146757-aa08-4263-aca3-15647c8c5d97 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:31.761912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:31.762807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:31.763638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 207/419] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:31.764299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.803751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.805361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.805863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.806355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] PATH_INFO: `/v3/projects/18a3d39073f24e449904bb491fbe8869/groups/de380a20db694174bbdaf16f0cc4232d/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.809525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] RBAC: Authorizing `identity:create_grant(project_id=18a3d39073f24e449904bb491fbe8869, group_id=de380a20db694174bbdaf16f0cc4232d, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:31.811207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:31.831252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:31.874632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 213/420] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Sep 1 16:14:31 2022] PUT /identity/v3/projects/18a3d39073f24e449904bb491fbe8869/groups/de380a20db694174bbdaf16f0cc4232d/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:31.875543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:31.885963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44471e07-6ea8-47da-aa6b-bf36af4f98d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:31.937670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:31.940111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:31.940586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:31.941049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:31.957100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:31.964456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:31.965425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:31.966239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 208/421] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:31.972430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.008966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2110c9-9a47-4408-942e-00ed423a607b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.059272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.060708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.061167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.062006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.063206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:32.064332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.066117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.066625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.079187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 214/422] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:32.088417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.090751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41601380-6689-4286-80d6-ddedf2bbb2e5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.115099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.137845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.138803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.139090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.139523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.140324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:32.140625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.143978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.154758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 209/423] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:32.165591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.234326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65427e07-89bf-47ba-a26b-f72b87869274 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.280910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.281752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.282225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.282794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.367062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.368753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.369107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2365a-ed8b-4d4e-a3e0-cc8eec8f5135 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.386443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.491371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 215/424] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:32.492641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.493266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.493547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.493819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.502627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.520856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.612852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.614347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.684488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d538b40-87be-4fa2-88c9-0437b64348cc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.713226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.758618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.760082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.760553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.761012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.762380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.764077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.771507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 216/425] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:32.772576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.784936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 210/426] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:32.786843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.863503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:32.928487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bca418-eac3-4a9c-803e-ec96a612614b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:32.964842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:32.965397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:32.965675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:32.965946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] PATH_INFO: `/v3/projects/bc0910ca971f4c5186644553060c147f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:32.967069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] RBAC: Authorizing `identity:get_project(project_id=bc0910ca971f4c5186644553060c147f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:32.968622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:32.971085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 211/427] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/projects/bc0910ca971f4c5186644553060c147f => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:32.977550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:32.986435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04783da-32b9-4d1c-b812-43055e5a8ab9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:33.035901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:33.035901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.035901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.035901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.036976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:33.038471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:33.040596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 217/428] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:33.046028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:33.107388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:33.169673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef4758-bbf3-4527-a80e-6724194561fd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:33.209855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:33.210515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.210760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.211036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.211844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=bc0910ca971f4c5186644553060c147f) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:33.212158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:33.213778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:33.221370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 218/429] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:33 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:33.243296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce5414-a8f1-47e1-b49b-eeafd93f3288 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:33.258035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:33.285006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:33.285661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.285937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.286207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.287358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:33.288811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881d9b0-ab70-4f91-9638-ca96a8e7a74c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:33.290669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 212/430] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:32 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:33.299333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:33.432184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c9d799-905a-450b-8a90-c20f58ec4e22 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:33.447141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:33.450623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f912-e2db-4397-be41-13144b6fe70e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.451253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f912-e2db-4397-be41-13144b6fe70e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.451744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a4f912-e2db-4397-be41-13144b6fe70e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.458390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 213/431] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:33.462971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953da2-1637-4fad-8b2d-dc79fa708656 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.462971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953da2-1637-4fad-8b2d-dc79fa708656 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.462971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953da2-1637-4fad-8b2d-dc79fa708656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.486830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:33.489006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.489690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.493279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.494317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=bc0910ca971f4c5186644553060c147f) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:33.494837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:33.496547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85890fc-19d3-4a8e-9731-2c64ef128b85 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:33.497787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-05953da2-1637-4fad-8b2d-dc79fa708656 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:33.502318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-05953da2-1637-4fad-8b2d-dc79fa708656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:33.510707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 219/432] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:33 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=bc0910ca971f4c5186644553060c147f&role.id=8488c378257a4f629184ae9773b882ac => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:33.596945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 214/433] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:33.859989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8827d-a343-4102-a1e7-e5b3c1526fd9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.861523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8827d-a343-4102-a1e7-e5b3c1526fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.862026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8827d-a343-4102-a1e7-e5b3c1526fd9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.863506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 220/434] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:33.869025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:33.869312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:33.869583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:33.889466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:33.890911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8827d-a343-4102-a1e7-e5b3c1526fd9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:33.891333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:33.977841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 215/435] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:34.016518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8827d-a343-4102-a1e7-e5b3c1526fd9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.056357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.056981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.057252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.057514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.069494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:34.075366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:34.076236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 01 16:14:34.077020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:34.078966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 221/436] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:34.085261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.134226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.196413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d57ec7-751d-4f5d-999a-6b54ddf819a1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.232139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.232823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.233105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.233384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.234050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:34.234348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.236371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.247440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 216/437] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:34.253908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.255372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.302125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.357810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.360749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d87ffd3-0749-4bce-b493-12c872473209 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.366778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602be2-9126-418b-bbfd-391e075b0864 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.395346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.395893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.396256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.396429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.401821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.402420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.402696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.402968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.406350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ec9a86f9-d43d-42d3-837e-2cc934786951 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:34.411892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:34.412860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:34.413682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 217/438] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:34.414352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954c3bb-bdd4-4e7f-a5eb-2f0dfa8a4a2b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.414430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954c3bb-bdd4-4e7f-a5eb-2f0dfa8a4a2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.414815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954c3bb-bdd4-4e7f-a5eb-2f0dfa8a4a2b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.416099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 218/439] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:34.418594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954c3bb-bdd4-4e7f-a5eb-2f0dfa8a4a2b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.478152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.479724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.512669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.523458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954c3bb-bdd4-4e7f-a5eb-2f0dfa8a4a2b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.557539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.558170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.558449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.558722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.559419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:34.559722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.561101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc20cf1a-9ae1-479b-9540-0e2b3b29a37f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.573613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 219/440] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:34.573613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.573613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.573613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.577328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 222/441] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:34.578240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.590948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:34.592177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:34.609105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.659205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 220/442] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:34.659966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.686905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2539e472-494d-40ab-a93a-cf9ba679050c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.726157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.726758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.727048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.727352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.730985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.759619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c05f94-237e-4efb-bf7d-14e7f0508850 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.793663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.794300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.794574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.794848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.795981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.797403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.799194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 221/443] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:34.805056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.806341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.808199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.827741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:34.908262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741968d-f666-4325-8448-9e3c2f9a9ada None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:34.921016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 223/444] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:34.922146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.945422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:34.946049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:34.946323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:34.946588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:34.947742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:34.949188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:34.950877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 222/445] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:34.957363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:34.959185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.023064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2fa27-1e55-4129-b5b2-08fae07691dd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.044400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.057040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7554b2-0cff-4916-9d66-d681992b2ce4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.057426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.058049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.058326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.058591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] PATH_INFO: `/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.059725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] RBAC: Authorizing `identity:get_project(project_id=cce8cfcae70e4b97b428d49b21a7acd3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.061159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.062994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 224/446] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/projects/cce8cfcae70e4b97b428d49b21a7acd3 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:35.064131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.092818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.093427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.093702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.093970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.094702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=184b9cf9fc5144009f8b26213f237b6c, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:35.095016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.096500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.103496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 223/447] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:34 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=184b9cf9fc5144009f8b26213f237b6c => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:35.103496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.177682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb0976-a42f-4c47-b8f3-8a3f11f41f2c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.180132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.215788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.216437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.216703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.216975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.227917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628f183-905c-4926-8a53-fe3c0d6a37c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.229139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:35.234049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:35.235024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:35.240120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 225/448] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:35.242604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.280789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.281425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.281701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.281970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] PATH_INFO: `/v3/groups/1a03050e0c884c9b8ab3d2998a47f741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.283953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] RBAC: Authorizing `identity:get_group(group_id=1a03050e0c884c9b8ab3d2998a47f741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.284931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.286486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 224/449] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/groups/1a03050e0c884c9b8ab3d2998a47f741 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:35.292560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.319404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.368608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff53aee-108c-41a5-a695-1d050311a9f0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.406372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.406980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.407983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.408264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.408939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:35.409240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.410632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.416935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a24f2e-505f-4d5f-b050-694ae65efd4e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.420228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 226/450] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:35.426658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.457523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.460962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.462073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.462736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.463215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.464087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] RBAC: Adding query filter params (group.id=1a03050e0c884c9b8ab3d2998a47f741, role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:35.464560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.465998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.473459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 225/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/role_assignments?group.id=1a03050e0c884c9b8ab3d2998a47f741&scope.project.id=cce8cfcae70e4b97b428d49b21a7acd3&role.id=8488c378257a4f629184ae9773b882ac => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:35.539620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74200b6c-8ec7-4fce-9231-c7d187475464 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.575852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.576464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.576737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.577001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.648384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.649839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.677503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.753852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 227/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:14:35.759663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:35.847059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:35.862353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685af9c-dbdc-4510-ac5a-138a50ea3eec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:35.896324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:35.896940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:35.897212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:35.897472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] PATH_INFO: `/v3/projects/18a3d39073f24e449904bb491fbe8869` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:35.898557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] RBAC: Authorizing `identity:get_project(project_id=18a3d39073f24e449904bb491fbe8869)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:35.900016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:35.901597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 226/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/projects/18a3d39073f24e449904bb491fbe8869 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:35.907927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:36.004706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16528411-dcf2-498e-b447-eeb75f2b9ba3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:36.029422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:36.038561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] PATH_INFO: `/v3/groups/de380a20db694174bbdaf16f0cc4232d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] RBAC: Authorizing `identity:get_group(group_id=de380a20db694174bbdaf16f0cc4232d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f50cf-4dcb-49e7-a1ab-5851e467697a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:36.045320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 228/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Sep 1 16:14:35 2022] GET /identity/v3/groups/de380a20db694174bbdaf16f0cc4232d => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:36.049754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:36.146184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d166b93-ab10-4c08-8502-cae1681d93c2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:36.179444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:36.180036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:36.180308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:36.180572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:36.181287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] RBAC: Adding query filter params (group.id=de380a20db694174bbdaf16f0cc4232d, role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=18a3d39073f24e449904bb491fbe8869) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:36.181585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:36.183006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ae16d-76bf-4e4e-a00b-50b0d39e7043 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:36.189298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 227/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Sep 1 16:14:36 2022] GET /identity/v3/role_assignments?group.id=de380a20db694174bbdaf16f0cc4232d&scope.project.id=18a3d39073f24e449904bb491fbe8869&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:38.550921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b547-b288-4d9d-844e-734d711366fb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:38.551736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b547-b288-4d9d-844e-734d711366fb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:38.551736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946b547-b288-4d9d-844e-734d711366fb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:38.553352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 229/456] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:38.558368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0b315-4474-4138-b4da-b9354de75161 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:38.558780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0b315-4474-4138-b4da-b9354de75161 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:38.558899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0b315-4474-4138-b4da-b9354de75161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:38.586828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f7c0b315-4474-4138-b4da-b9354de75161 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:38.595007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7c0b315-4474-4138-b4da-b9354de75161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:38.604403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:38.661168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18504-fa67-4b4d-8c5a-73b11424003e None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:38.661488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18504-fa67-4b4d-8c5a-73b11424003e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:38.661726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18504-fa67-4b4d-8c5a-73b11424003e None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:38.667387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 230/457] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:38.671988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63f0b-816f-4b14-aed8-bc07e21ecde1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:38.672273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63f0b-816f-4b14-aed8-bc07e21ecde1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:38.672552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe63f0b-816f-4b14-aed8-bc07e21ecde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:38.676813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 228/458] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:38.712140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bbe63f0b-816f-4b14-aed8-bc07e21ecde1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:38.713420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bbe63f0b-816f-4b14-aed8-bc07e21ecde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:38.809038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 231/459] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:39.005180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3885e-5a47-41c0-8b60-2d9277b60a19 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.005543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3885e-5a47-41c0-8b60-2d9277b60a19 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.005732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3885e-5a47-41c0-8b60-2d9277b60a19 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.007369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 229/460] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:39.012424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.012929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.013397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.019433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3885e-5a47-41c0-8b60-2d9277b60a19 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.039475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:39.040989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:39.129060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3885e-5a47-41c0-8b60-2d9277b60a19 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.137697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 232/461] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:39.167164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.167811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.168085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.168349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.180973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.182571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:39.191216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:39.200331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:39.201231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:39.202081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:39.202081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.202081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:39.207256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 230/462] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:39.208839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.220863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.315511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.401265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54aa7d-edc0-427c-8ca1-d99ba16c08a5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.414718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fae14c3-0fc8-48ee-b853-93eb86ed374a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.455406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.456013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.456291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.456568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.457220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:39.457523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.458893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fabdb-3fd1-4026-8ca8-b2ad646f2848 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:39.462614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.463271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.463550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.463818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.471265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 233/463] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:39.472200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3b3e0-6575-4a9f-b6cd-11f75c3658c3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.472485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3b3e0-6575-4a9f-b6cd-11f75c3658c3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.472783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3b3e0-6575-4a9f-b6cd-11f75c3658c3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.474243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 234/464] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:39.474945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3b3e0-6575-4a9f-b6cd-11f75c3658c3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.479059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.480568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:39.495556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:39.496467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:39.497343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:39.497343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.497343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:39.497343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 231/465] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:39.497968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.554114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.587283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3b3e0-6575-4a9f-b6cd-11f75c3658c3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.601314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.609401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77089ef-9e5e-4dda-87bc-63d0d495ad7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.621892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.622517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.622791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.623059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.634982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.636571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:39.646190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.646803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.647080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.647380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.648075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.649518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-de6b4900-370f-437d-a803-cc23d797b0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:39.651526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:39.652355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:39.653177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:39.653177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:39.653177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:39.653177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 235/466] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:39.653858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.654114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.654385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.675917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:39.677361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:39.743242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 236/467] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:39.744129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.764023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 232/468] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:39 2022] POST /identity/v3/users => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:39.765436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:39.924398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.934755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d9e7-c021-42b6-802a-bb9fc09ec9c5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.964483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae71660-78dd-4126-b987-9e2afc397fcd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:39.973094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:39.984703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:39.985501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:39.985775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:39.986048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:39.986959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:39.987328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:39.988694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.004714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 237/469] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:40.009011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.009630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.009903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.010182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.010977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:40.013813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.013913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.013988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.028881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 233/470] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:40.034957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.132329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0657a30-c584-4d37-aad1-c3fb67ad9aff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.149276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a8f54-532c-43b1-9ada-bff4ffc463fc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.169060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.169747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.170138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.170405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.171062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.172618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.184199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.184741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.185019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.185288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.185929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.187374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abf5d2-d1ce-4f80-961a-ca9fcb77cc9b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.254265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 238/471] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:40 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:14:40.255374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.285051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:40.289259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 234/472] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:40 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:14:40.291804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5941c6-d063-4623-9329-64ce62a1eb2a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.292412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5941c6-d063-4623-9329-64ce62a1eb2a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.292951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5941c6-d063-4623-9329-64ce62a1eb2a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.295675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 235/473] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:40.301059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.301593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.301861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.347094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:40.347457 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:40.386442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:40.449344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e8c88-082d-44ed-914c-589ff7973820 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.476084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 236/474] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:40.507496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.508189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.508467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.508780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.520093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.521544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:40.537308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:40.538210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:40.539214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:40.539214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:40.539214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:40.539214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 239/475] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:40.544309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.653963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4779877-57be-476c-9439-0c4493a696b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.690547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.691193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.691481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.691758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.692411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:40.692717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.694111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.706546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 237/476] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:40.712827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.778900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.803542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:40.817959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:40.884486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd513b09-14da-45d3-bf7a-fc4393f6feae None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.919949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.920568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.920844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.921114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.921807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.923838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:40.926068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f465720-4342-4d3c-b14d-fc7812adbbaa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:40.962242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:14:40.962242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:14:40.962242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [parameters: {'id': 'b6d686397d1449d4aab0f44b657102bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:14:40.962242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:14:40.979639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:14:40.980334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return ref.to_dict() Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:14:40.981340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:14:40.982430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:14:40.983674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:14:40.984811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:14:40.986442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': 'b6d686397d1449d4aab0f44b657102bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:40.987386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 01 16:14:40.988192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:40.989051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 238/477] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 16:14:40 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:40.989051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:40.990891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:40.990891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:40.990891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:40.990891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:40.997520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:41.088084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 240/478] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:40 2022] POST /identity/v3/users => generated 249 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:41.089923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:41.116405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fab9a-eea6-41f7-a209-da7d60ada0f0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:41.154416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:41.155382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:41.155831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:41.156245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:41.169033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b8e44f58-5d33-4d06-a7af-28926583c075 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:41.175220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:41.176679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 01 16:14:41.177646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:41.179919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 239/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:14:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:41.186748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:41.329188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd337c-4dae-478e-9497-428ffd0e5b65 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:41.349081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:41.364491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:41.368638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810628c5-bb5f-4494-acb9-9847e6671ef4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:41.383299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 241/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:42.293343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929137-a035-4622-809f-e3eafa2a9e30 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.294191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929137-a035-4622-809f-e3eafa2a9e30 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.294636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89929137-a035-4622-809f-e3eafa2a9e30 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.296207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 240/481] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:42.302383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3eb0-2b72-4218-907a-11ac0d27fbeb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.302818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3eb0-2b72-4218-907a-11ac0d27fbeb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.302951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b3eb0-2b72-4218-907a-11ac0d27fbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.328059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-705b3eb0-2b72-4218-907a-11ac0d27fbeb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:42.329291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-705b3eb0-2b72-4218-907a-11ac0d27fbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:42.412327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 242/482] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:42.617711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4b9e-a028-4cfb-98b2-05f0c4f18c57 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.618035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4b9e-a028-4cfb-98b2-05f0c4f18c57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.618264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d4b9e-a028-4cfb-98b2-05f0c4f18c57 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.619901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 241/483] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:42.625006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a4821-0633-4b1d-ab2b-b92e430ce2a4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.625363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a4821-0633-4b1d-ab2b-b92e430ce2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.625637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a4821-0633-4b1d-ab2b-b92e430ce2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.652086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5e4a4821-0633-4b1d-ab2b-b92e430ce2a4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:42.653276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e4a4821-0633-4b1d-ab2b-b92e430ce2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:42.741780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 243/484] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:42.753939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d5f3-3ff7-4d9c-9a42-17e889afe398 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.754253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d5f3-3ff7-4d9c-9a42-17e889afe398 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.754476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172d5f3-3ff7-4d9c-9a42-17e889afe398 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.756083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 242/485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:42.761632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:42.761913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:42.762177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:42.791209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:42.792400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:42.838930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:42.843441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172d5f3-3ff7-4d9c-9a42-17e889afe398 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:42.883737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 244/486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:43.007832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172d5f3-3ff7-4d9c-9a42-17e889afe398 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.043878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.044554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.044838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.045109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.047914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.056371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:43.061417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:43.064407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:43.065208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 243/487] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:43.071577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.094317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.233150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd910-5580-4ca7-b2c7-cf381cce432d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.264265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c07513-afbb-4fac-82eb-7ab5568d44f5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.306254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.306782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.307178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.307178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.307492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.307600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.307671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.311173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.311294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:43.311294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.311294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.319321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 245/488] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:43.320520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.321275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:43.327646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:43.328593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:43.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 244/489] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:43.331882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.358335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.407402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.450282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac5030-750c-4453-83cf-5b8610c72d47 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.452803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c00b-cd89-4283-8cc3-a19cbc1b9bcf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.489732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee262-1f1d-4794-9692-080121496137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.490438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee262-1f1d-4794-9692-080121496137 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.490815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee262-1f1d-4794-9692-080121496137 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.491203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eee262-1f1d-4794-9692-080121496137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.497249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.498088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.499030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.499470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.503428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eee262-1f1d-4794-9692-080121496137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.505013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eee262-1f1d-4794-9692-080121496137 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.513688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:43.519556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:43.520631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:43.521549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-34eee262-1f1d-4794-9692-080121496137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:43.523315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:43.524289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:43.525314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:43.525314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:43.525314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:43.525693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 246/490] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:43.526590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.531743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 245/491] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:43.532787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee262-1f1d-4794-9692-080121496137 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.646385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eee262-1f1d-4794-9692-080121496137 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.656153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.657012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.665462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e6194-5f27-4320-a531-e0eef47be9db None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.683930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.684541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.684817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.685092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.685752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:43.686053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.687760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.701075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 246/492] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:43.701075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.711118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.712275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.712948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.713367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.714349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:43.715353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.717651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc33e2-402e-48ee-9203-49958de3d9f2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.728904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 247/493] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:43.730447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c5cb-a91f-4f0f-82af-0d9806c4fbf4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.731169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c5cb-a91f-4f0f-82af-0d9806c4fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.731837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4057c5cb-a91f-4f0f-82af-0d9806c4fbf4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.733426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 248/494] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:43.735222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c5cb-a91f-4f0f-82af-0d9806c4fbf4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.804933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20f99a-d210-4519-b043-4a69da7f889d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.840273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.840873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.841151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.841423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.842059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:43.842360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.843763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.846475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4057c5cb-a91f-4f0f-82af-0d9806c4fbf4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:43.855957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 247/495] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:43.857060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:43.885560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:43.886180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.886454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.886724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.889346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.900988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:43.902638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8d7cc26a-18dc-466a-9cb5-0593a5e38cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:43.918057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:43.919001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:43.920197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:43.920197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:43.920197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:43.920197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 249/496] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:43.920408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:43.924093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:43.924093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:43.941578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:43.942767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:43.944932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:43.971441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0b1d-f303-4742-b1e8-73cfa1bce0a6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.005982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.006611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.006887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.007174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.009494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 250/497] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:44.010313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.022225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.023908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:44.040359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:44.041435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:44.048146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:44.048146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:44.048146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:44.048146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 248/498] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:44.048146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.123738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c66032-5977-4444-9aaf-03bb618d9594 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.135870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.154718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec3688-f929-428a-90ec-e09dae1627e7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.159038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.159695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.159969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.160234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.167649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.169079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.171215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-44d452d2-df59-4510-b8b1-d488ccbca95a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:44.184804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:44.185718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:44.186630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 251/499] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:44.187191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869fa0-5b01-4242-860c-47f8560fa60b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.189579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869fa0-5b01-4242-860c-47f8560fa60b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.189579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73869fa0-5b01-4242-860c-47f8560fa60b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.190065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 252/500] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:44.190759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73869fa0-5b01-4242-860c-47f8560fa60b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.202663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.203311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.203591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.203932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.204597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:44.204918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.206682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.218918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 249/501] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:44.219860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.306941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73869fa0-5b01-4242-860c-47f8560fa60b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.331331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913c914-9c12-48cb-b7e1-44fa90a4a716 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.340604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.341203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.341487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.341758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.342402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:44.342701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.344103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998acaf-49fe-4933-87eb-6b7b35a83f3e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.356102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 253/502] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:44.356749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.357031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.357302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.369373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.370007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.370316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.370585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.371377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:44.371678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.373150 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.376505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:44.377699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:44.406845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.411386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 250/503] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:44.412187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.432622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.445687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 254/504] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:44.446483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.522352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f78d8-b69f-478f-909e-240165110bc5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.551093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93b3bc-adc4-4829-a014-d8200ad7d60c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.558725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.559387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.559664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.559928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.568807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.570381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:44.583991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:44.588207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:44.589231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:44.589933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.590038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 251/505] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:44.590117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.590197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.590283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.590283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.600482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.601926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:44.618001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:44.618984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:44.620170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:44.620170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:44.620170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:44.620170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 255/506] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:44.624490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.657956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.667504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.702722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fa7d-b043-4475-8b07-c2d2d4fc40f5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.727551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cfc30-2401-49c6-9e40-6df9a4c97eaa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.737029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.737659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.737935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.738202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.746046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.747530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:44.759299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:44.760355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 252/507] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:44.761561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.763610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.764217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.764499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.764775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.833132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.834584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.864979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93e391-bb01-437f-a5cb-e782ba245cb4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:44.876568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.884916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:44.898386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:44.899070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:44.899377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:44.899652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:44.900381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:44.900683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:44.902113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:44.918752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 253/508] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:44.919532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:44.935290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 256/509] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:44.936132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.020048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ed3ae-2bee-4bc7-8781-72ac01686fd3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.039965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93629c-8829-47c3-8eb9-87b5056ba2e6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.053187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.053792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.054066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.054333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.054969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:45.055297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.056668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.068347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 254/510] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:45.069158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.078747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.079715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.079985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.080250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.088226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.091103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:45.095944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:45.105430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 257/511] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:45.106203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.108226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.169045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd338b14-0b13-46aa-b26e-6599890f20e3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.198269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888535f-5eb0-4d16-a294-766130e04463 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.202661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.203312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.203594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.203913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.204816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:45.205124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.206569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.226038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 255/512] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:45.226866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.232720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.233325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.233600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.233865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.300110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.302707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.304188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.329729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.335367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2a581-8f66-4caf-a1e7-135157a14b99 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.369417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.370046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.370325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.370597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.381831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.383325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:45.400778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:45.402689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:45.406299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:45.406299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:45.406299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:45.406299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 256/513] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:45.406714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 258/514] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:45.406714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.407515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.499625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.502986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f586d55-c3c1-4904-9585-5b6cbfabdc93 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.505587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ff7bc-c050-41f3-bca0-d32f4135fa37 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.538525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.539426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.539575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.539838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.540025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.540249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.540346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.540556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.541193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.541279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:45.541674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.542624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.543797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.557177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 259/515] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:45.558020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.624063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 257/516] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:14:45 2022] POST /identity/v3/users => generated 247 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:14:45.635585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.645527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.682856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95708946-900a-4c4b-bbd8-3f5e4e083887 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.720891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.721513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.721788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.722055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.754882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.762851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611b3bf-3613-4437-89e3-0af144c265a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.803705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.804061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.804680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.804953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.805224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.805868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:45.806075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.806172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.807596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.822166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 258/517] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:45.823025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.887261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:45.920974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 260/518] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:45.928270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:45.934414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063b79f-8033-4add-9a46-094194e71ffb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:45.971226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:45.971867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:45.972148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:45.972419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:45.983955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:45.986082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:45.992366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:46.002400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.010438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:46.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:46.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:46.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 259/519] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.040177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d5126-4e20-4ccb-ae4c-29bc4f3b3930 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.077731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.078368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.079073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.079073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.090546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.091953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:46.108286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.109225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:46.110658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:46.110658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.110658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:46.110658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 261/520] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.110998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.122625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabc82e-2a75-407b-a45b-66ae88528e44 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.141411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.163749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.164395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.164680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.164947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.172199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.174332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:46.187868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.189285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:46.190303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:46.194258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 260/521] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.195009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.253876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811678-e48c-49b9-8fa8-64ad696ae680 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.273640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.295741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.296385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.296663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.296938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.311144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.312640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.326860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c439c6-4649-4ca6-a1f2-a8ef3a222a31 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:46.330346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.331485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:46.332333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:46.332333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:46.332333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:46.332333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 262/522] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.333100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.373053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.373712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.373920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.374186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.374834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:46.375319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.377423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.390089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 261/523] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:46.390714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.440026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.476020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04846712-c5aa-4d5f-8266-769128e68dbc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.527635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.528172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.528448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.528716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.529240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ad038-43fc-45a5-bd15-bb2d674ea9a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.529409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:46.529661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.531050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.549362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 263/524] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:46.549362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.574996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.576081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.576081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.576336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.577223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:46.577449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.581833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.585829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.600984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 262/525] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:46.601912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.682402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef8466-4509-4e97-bf9a-ef5b2c8fa895 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.725003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.725517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.726880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.727228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.727501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.728494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:46.728810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.730200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.731589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6d8bb-fd93-43ce-af0c-a1f7fc68b325 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.751253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 264/526] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:46.751253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.786169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.786732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.787013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.787317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.794502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.795986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:46.808635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.809563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 263/527] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.816586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.878916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:46.896114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652b939-6f19-4344-b2a5-a6cda96a74bb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.939461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.940169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.940496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.940724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.944100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55565180-49bc-4438-bb2c-2c3136c0885f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:46.953205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.955393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:46.968687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:46.969859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 265/528] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:46.970973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:46.990623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:46.991269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:46.991550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:46.991824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:46.992555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:46.992859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:46.994315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.007195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 264/529] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.007195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.040823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.118706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83801620-2342-4b9b-8a8b-b72255498723 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.143226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869aab-3481-4e82-97aa-d6da54db2697 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.169346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.170436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.170436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.170707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.179922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.181464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.185551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.186340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.186769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.187261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.188762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:47.189238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.190826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:47.195093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:47.196172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:47.197207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:47.202722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 266/530] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:47.202722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.211715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.217545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 265/531] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.218781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.319912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50df8de-ef42-42f9-9092-7f0eddbc3fad None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.322926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.345164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1261a7f9-9559-4dac-b5a9-fbe7baf13a6c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.355215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.356031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.356141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.356408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.357141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:47.357444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.358927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.384989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.386350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.386829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.387351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.388249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:47.388752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.390400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.403212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 267/532] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.403212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.410435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 266/533] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.411780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.466514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.506436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20707c73-9bd3-47a6-9ec9-163a4ba39350 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.523598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d0873-d291-4dc1-979c-34518335d675 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.552878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.553828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.554341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.554811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.555824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=4a081c08049447949477e73e0891f1e8) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:47.556330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.558052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.564912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 268/534] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/role_assignments?user.id=4a081c08049447949477e73e0891f1e8&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.565991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.572024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.574036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.574657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.575559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.599301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.680912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.682569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.709781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170455-3d0d-45af-9b69-e93f264e7882 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:47.744922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.759421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:47.760391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.760737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.761113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.762156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=e346b0a022ea4218a4bd10c7e6fe6d91) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:47.763333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:47.764935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3859e-b10b-4f09-ae08-44dbb4a12134 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:47.773672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 269/535] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/role_assignments?user.id=e346b0a022ea4218a4bd10c7e6fe6d91&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:47.779270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591f9b-a874-4a97-812b-8f3a9322d16a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.779270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591f9b-a874-4a97-812b-8f3a9322d16a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.779270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a591f9b-a874-4a97-812b-8f3a9322d16a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.779270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 270/536] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:47.784063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:47.784348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:47.784761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:47.807260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 267/537] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:47.811540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:47.812853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:47.816409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:47.897996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 271/538] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:47.908388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:47.992802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b5a72-6cdc-4f0c-bc08-5aa568314413 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.032573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.033214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.033487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.033753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.050142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:48.051889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:48.075047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:48.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:48.076847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:48.076847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:48.076847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:48.076847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 268/539] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:48.082581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.223977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b919a6-4660-4181-8df9-075652f7f09e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.253981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:48.275563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.276174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.276451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.276726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.277373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:48.277671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:48.279045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:48.291474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 272/540] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:48.297279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.310635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.451537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:48.460800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c996b977-95b1-4644-a40c-466dc25c3c59 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.466067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e8b91-724a-4319-a7c3-b72abe74dbe2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.497756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.498700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.499196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.499661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.510902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:48.511499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.511718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.511946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.512215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:48.517639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:48.518493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:48.519308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:48.519895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:48.520897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:48.522146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 273/541] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:48.532808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:48.533240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:48.534037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:48.535260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 269/542] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:48.540196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.620337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:48.675764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167cf63-a334-45b3-ad8c-58c159eaeef8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.722532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284e267-0cd5-4c1f-95a8-5cf1d019166d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.748595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.749170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.749444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.749714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.750434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:48.750892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:48.752574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:48.769136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 274/543] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:48.780174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.783722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.784347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.784624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.784893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:48.786452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:48.786766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:48.788248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:48.803228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 270/544] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:48.816694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:48.845363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:48.946220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef25fd8-fc6a-40ef-9889-2206df18b9ba None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:48.948685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:48.989472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:48.990120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:48.990397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:48.990715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.011753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.013145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:49.035595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:49.036721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:49.037950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:49.037950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:49.037950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:49.037950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 275/545] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:49.038267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0c2f-2dc0-4bc1-a3ab-aa70a5c172fe None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.042603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:49.087099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.087748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.088032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.088309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.089045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=793f922871f04a6480c571a073534d3f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:49.089354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.090811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8cdd7-e7ae-4c8a-aeb9-53f368963111 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.111669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 271/546] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:48 2022] GET /identity/v3/role_assignments?user.id=793f922871f04a6480c571a073534d3f&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:49.139637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bdb8-3223-4df6-8532-48ac62f43454 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.139937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bdb8-3223-4df6-8532-48ac62f43454 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.140335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4bdb8-3223-4df6-8532-48ac62f43454 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.142577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 272/547] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:14:49.149271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.149678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.149835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.192301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:49.193580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:49.206757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d05cbe-57f8-4747-927a-fc21d2170eb0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.246561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.246913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.247255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.247580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.248248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:49.248590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.250292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.265584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 276/548] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:49.274278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:49.313000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:49.344934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 273/549] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:49.427804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07accc43-e102-4626-9e3b-ab74bc0facce None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.477014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.477703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.477983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.478250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.489573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.491086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:49.506545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:49.508812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:49.510950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 277/550] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:49.513339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:49.527836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:49.653441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0dcb3-3f44-4ca5-937d-b01df33e0243 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.697143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:49.699288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.699917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.700195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.700469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.701210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:49.701514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.702956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.722525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 274/551] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:49.727963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:49.794751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:49.818157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:49.835972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4361eb6-9662-42ab-b5f8-d0b8a0d20ba2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.866957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081812fb-2040-40f4-994b-152da8a1a063 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:49.872912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.873510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.873792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.874057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.929285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:49.929933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:49.930215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:49.930483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:49.945968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:49.947519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded97ac-86cf-4062-82f3-e3f7ef9a9a7d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:49.948679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:49.954576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:49.956733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:49.958747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:49.964126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 278/552] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:49.969183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.063634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.137974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 275/553] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 01 16:14:50.144584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a75f-b981-4401-a92a-de17abfe7a5e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.144893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a75f-b981-4401-a92a-de17abfe7a5e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.145173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a75f-b981-4401-a92a-de17abfe7a5e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.146499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d21030-b189-4ea5-9336-3d33a3f68782 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.146747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 276/554] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:50.147600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a75f-b981-4401-a92a-de17abfe7a5e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.187038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.187703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.187981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.188570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.189235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:50.189542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.190921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2915c08-6625-4dbf-8465-ce0bc1b79ca8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.193489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.201187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 279/555] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:50.207477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1cde8-8710-4449-b44c-7599f2466dcd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.207477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1cde8-8710-4449-b44c-7599f2466dcd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.207477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1cde8-8710-4449-b44c-7599f2466dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.222226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-10d1cde8-8710-4449-b44c-7599f2466dcd None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:50.223451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-10d1cde8-8710-4449-b44c-7599f2466dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:50.263749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a75f-b981-4401-a92a-de17abfe7a5e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.300017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 280/556] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:50.301285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f9c5-9ce0-4a9c-b997-2254b2f68b57 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.301831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f9c5-9ce0-4a9c-b997-2254b2f68b57 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.302102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6f9c5-9ce0-4a9c-b997-2254b2f68b57 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.307269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 281/557] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:50.307269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6f9c5-9ce0-4a9c-b997-2254b2f68b57 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.310639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.312359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.316143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.316143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.327008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.330695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8b8e7b0c-fc36-4707-a37c-b77828f20e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:50.364256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:50.365251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:50.366119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:50.366119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:50.366119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:50.366119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 277/558] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:50.368655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.369197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.369608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.400797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:50.403296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:50.418565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.432289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6f9c5-9ce0-4a9c-b997-2254b2f68b57 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.465097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.473848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.474597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.474976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.475389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.485875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.487499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.499791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 278/559] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:50.499791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:50.514299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:50.515685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:50.517261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:50.517261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:50.517261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:50.519619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 282/560] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:50.524976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.652665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4a37-8a16-495b-8a58-9bd5a1b389c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.689831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f726580-3459-4379-b50d-0e91cb0e2a81 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.735490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.736117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.736397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.736499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.736696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.737032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.737315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.737411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:50.737583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.737767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.738237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:50.738537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.739006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.739914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.753851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 279/561] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:50.754699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.755176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 283/562] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:50.763266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.770573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.820988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:50.871461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348559eb-5157-4b16-8bca-41127372fa7d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.886721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5a055-2d00-4423-b205-5afe8ce14719 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:50.908645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.909633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.909929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.910203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.920758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:50.925151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:50.926015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:50.926749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:50.926883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:50.927921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:50.928768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 280/563] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:50.929519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:50.929968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:50.941388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:50.942939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:50.955113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:50.956220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 284/564] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:50.957138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.004696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.039713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b093e7-2741-4b0b-ba03-2f7cd4fcc49a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.068275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.071419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339bef5-2b21-464f-af26-996ff5e01594 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.075094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.075762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.076041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.076311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.084161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.085649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:51.098559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:51.099500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:51.100407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 281/565] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:51.100887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.113029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.113662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.113988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.114518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.126290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:51.131315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:51.132244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:51.133069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 285/566] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:51.133793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.212058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf195c2f-34a7-4258-964e-b215d31ac780 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.243673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24169-debd-437d-9f6e-15badf508316 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.244240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.248957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.249568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.250189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.250189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.250805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.251192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.252587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.261576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 282/567] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.262481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.276653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.277257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.277536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.277877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.278616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.278919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.280396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.299353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 286/568] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.300160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.319991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.365273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e130b-69cf-4d32-92e2-234f39728613 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.398420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53856f-be99-4371-a3f5-f4c0f1be88a4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.401483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.402083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.402361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.402630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.403370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.403671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.405115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57528db-a563-4f62-a1fd-bc8a4218c725 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.417788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 283/569] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.418946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd100fb-b366-4198-90db-6d4826b37021 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.419273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd100fb-b366-4198-90db-6d4826b37021 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.419541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd100fb-b366-4198-90db-6d4826b37021 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.423215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 284/570] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:51.423215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd100fb-b366-4198-90db-6d4826b37021 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.436672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.437279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.437556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.437826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.438523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.438824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.440235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.448111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 287/571] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.448878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.472239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.523422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd100fb-b366-4198-90db-6d4826b37021 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.530959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.551003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ff269-73b4-4db4-8a55-f533bce42aad None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.558045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.558670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.558951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.559253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.569128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.570645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:51.585690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:51.592125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:51.593199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:51.593199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.593199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:51.593380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.593480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 285/572] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:51.593556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.593556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.593556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.593775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.598671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.600125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-23a22333-8097-440d-9e8c-6d7372a83268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:51.614453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:51.615613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:51.616695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:51.616695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.616695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:51.616695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 288/573] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:51.618500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29714c6-d034-4068-a991-305b72a6876e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.618786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29714c6-d034-4068-a991-305b72a6876e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.619064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29714c6-d034-4068-a991-305b72a6876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.636764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f29714c6-d034-4068-a991-305b72a6876e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:51.637968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f29714c6-d034-4068-a991-305b72a6876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:51.688292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.691048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcabd2-7734-4d11-8041-b17cf378fa25 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.706104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 289/574] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:51.706758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29714c6-d034-4068-a991-305b72a6876e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.735550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.736104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.736381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.736652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.737380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=d247a8be3add4804b86dcc0cd4cb72e3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.737686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.739130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.759192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 286/575] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/role_assignments?user.id=d247a8be3add4804b86dcc0cd4cb72e3&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.759192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.769544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:51.816440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29714c6-d034-4068-a991-305b72a6876e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.856981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.857909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.858289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.858767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.870763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.872390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.880771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948af26a-fe9f-4567-84f1-9aa57d6282bf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:51.898421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:51.899659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:51.900727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:51.900727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:51.900727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:51.906117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 290/576] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:51.906117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.938612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:51.938612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:51.938612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:51.938612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:51.939069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:51.939069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:51.939069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:51.954868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 287/577] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:51.955935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:51.986246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.025048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1679e-5a88-4d77-970f-c0e8acb7fab2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.041927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.065494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.066198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.066537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.066873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.067611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6768db-eb3f-4d86-860c-6b7bcf361998 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.067739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.068061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.069503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.082286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 291/578] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.083124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.115783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.117122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.117609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.118077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.118814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.119202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.120733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.137815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 288/579] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.137815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.195164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbeb-7e44-4c4e-bf66-2bb05c72f95a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.229126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.235451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.236185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.236519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.236963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.244362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.245810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.247802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f3d0a-4cd5-43c6-ba46-054d9ad44b31 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:52.259465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:52.260436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:52.261358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:52.263455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 292/580] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:52.265110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.289312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.289949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.290228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.290499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.301102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.303276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:52.308211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:52.311910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 289/581] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:52.312811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.379890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e331562d-66a4-497a-abb9-02dcc0a4a93d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.415673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31364b01-b764-4cb4-8d03-57dbe6066e84 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.417254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.418539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.418539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.418539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.425823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.427542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:52.440174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:52.441053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 293/582] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:52.447366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.452719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.453353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.453631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.454409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.461792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.463277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:52.475586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:52.476510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 290/583] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:52.477452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.482257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.552283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.558804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e88a056-dc31-4521-a1d9-2876bbd695d5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.591777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b63435-d4e8-42c1-9e44-93ddc1970894 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.595359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.596565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.596901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.597187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.597933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.598237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.599725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.618736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 294/584] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.619873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.633700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.634312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.634588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.634856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.635644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.635945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.637313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.650970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 291/585] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.650970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.716711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.745667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca88e68-1f84-4ba4-afd2-43f31054f098 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.760190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5918e2d-59c7-48cd-9872-32131a14bb07 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.782965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.783938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.784215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.784481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.785326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.785950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.787833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.795827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.796434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.796708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.796974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.797854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:52.798402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.799867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.809269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 295/586] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.810048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.821465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 292/587] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:52.823096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:52.826522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.920780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50a2de-177e-446c-af25-a9f436cba00f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.933858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121591cc-6013-4ea4-85d7-f37b6f3868bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:52.958060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:52.959300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.960058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.960333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.960754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.969035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:52.970793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:52.971199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:52.971564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:52.972428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.973925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.982997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:52.984547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:52.988766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:52.989840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:52.990917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:52.990917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:52.990917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:52.994706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 296/588] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:52.995704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:53.002774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:53.003944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:53.005117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:53.005117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:53.005117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:53.006045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 293/589] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:53.011072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.064483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.107094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3344ad1-c71b-4222-a32e-0bf89fbe4ad3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.115617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a02ae-8b1b-4cd1-9094-2b550b174429 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.144732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.145396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.145672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.145936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.153151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9764125-5325-4676-9407-0d97f597449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.153897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9764125-5325-4676-9407-0d97f597449c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.154277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9764125-5325-4676-9407-0d97f597449c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.154649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9764125-5325-4676-9407-0d97f597449c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.162425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=68764b02b584408090a0c12a06b50445, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.163906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.165938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9764125-5325-4676-9407-0d97f597449c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.167482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9764125-5325-4676-9407-0d97f597449c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e9764125-5325-4676-9407-0d97f597449c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:53.182863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:53.184048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:53.185108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:53.185108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:53.185108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:53.185108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 294/590] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:53.185108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9764125-5325-4676-9407-0d97f597449c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.216040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.231186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 297/591] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:52 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:53.232118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.323827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9764125-5325-4676-9407-0d97f597449c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.360888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a61ebf-d29a-42e1-9e60-4a5d95785d18 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.375096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.376052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.376476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.376893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.378685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.378871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:53.379572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.381873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.395227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 295/592] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:53.396333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.409526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.410144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.410422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.410692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.411498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:53.411798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.413179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.428691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 298/593] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:53.429551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.514321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea75684-6f89-4ce6-8e9a-06626cfd421d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.547705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.552768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.553383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.553657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.553924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.554577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:53.554964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.556472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.558722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0fcb84-2858-45f5-9c0a-d1fe4ac83e7b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.575729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 296/594] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:53.576908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.600351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.601033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.601361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.601638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.611952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.613953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:53.629331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:53.630387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:53.631644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:53.636303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 299/595] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:53.637234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.683565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.694978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282a18c-6357-4a3c-91de-c1a675405f66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.732901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.733522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.733807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.734077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.742472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.743952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.754225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efadc80-23c6-4d41-9099-c391a08cb364 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:53.756922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:53.757861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 297/596] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:53.758788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.772294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.793696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.794323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.794597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.794866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.803107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.804558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-64b3a7b3-603c-48d3-b33a-e3c0a567efcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:53.816311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:53.817276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:53.818250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 300/597] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:53.818727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e017c4-f8c9-4889-8b4b-f9b67d1f5151 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.818803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e017c4-f8c9-4889-8b4b-f9b67d1f5151 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.819316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e017c4-f8c9-4889-8b4b-f9b67d1f5151 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.823277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 301/598] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:53.823277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e017c4-f8c9-4889-8b4b-f9b67d1f5151 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.873106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f10ed0-434e-4f7e-b132-e1f724de3401 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.909581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.910195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.910474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.910746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.911484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:53.911791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.913231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:53.926017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 298/599] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:53.927401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:53.934009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:53.944115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e017c4-f8c9-4889-8b4b-f9b67d1f5151 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:53.995758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:53.996359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:53.996700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:53.996987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:53.997712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:53.998021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:53.999469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b74c29-0ecf-4db6-96f3-209a82a88fd1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.004927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.018741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 302/600] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:54.023286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.023286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.023286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.038622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:54.039888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:54.043227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35669-8bb2-4f24-830e-ef6dbe453b98 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.078024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.078655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.079673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.079996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.080686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:54.080990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.082422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.095189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 299/601] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:54.100675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.112869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 303/602] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:54.114114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.224151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.228735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e5019-fccb-4088-a6a3-912e465eb83f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.252547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc4731-32d5-48f6-89ad-bebd0d55112c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.271972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.283690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.284356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.284633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.284903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/4a081c08049447949477e73e0891f1e8/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.297469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.298297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.298656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.298935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.308763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=4a081c08049447949477e73e0891f1e8, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.310096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.311036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.311578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:54.327831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:54.330800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:54.332047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:54.332047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:54.332047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:54.332047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 304/603] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:54.335172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.385477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 300/604] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:54 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/4a081c08049447949477e73e0891f1e8/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:54.386377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.471369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4068db-367b-4ea8-8c46-da29089fa7ff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.511524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f6512-81f6-4eb7-9776-b7f764642302 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.523697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.525346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.525633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.525903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/e346b0a022ea4218a4bd10c7e6fe6d91/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.529521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.545947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=e346b0a022ea4218a4bd10c7e6fe6d91, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.547954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.555570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.556472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.557002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.557431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.558085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:54.558383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.559786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.573156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 301/605] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:54.573998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.620161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.660562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 305/606] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:54 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/e346b0a022ea4218a4bd10c7e6fe6d91/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:54.666352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.689674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329673-7168-4156-b49d-83cbf44a0c8c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.730159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.731048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.731524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.731943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.745587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:54.752344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:54.753397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:54.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:54.757073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 302/607] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:54.764663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.838987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cf9cd-39ba-42ef-97f9-d951a5e4e8ec None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.885758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:54.887029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:54.887547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:54.888024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:54.899601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:54.902216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:54.916588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:54.917524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:54.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 306/608] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:54.923769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:54.945745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:54.950943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953cc76-7b58-4c96-b638-6dc528fbb4a6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:54.978562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:55.000368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.002269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.003339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.003908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.005116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:55.005632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.008220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.023355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 303/609] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:55.025142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:55.057386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90ea09-4470-40c1-9602-e39be77f8c9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:55.135307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.135926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.136201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.136471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.138623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:55.138934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.140410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.170116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 307/610] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:55.189662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:55.231514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeb40d-fda7-4c35-9b30-288c1c5da9ad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:55.333106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.333798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.334077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.334350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.346798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa86ec9-3438-4add-addb-4a2c41c76610 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:55.349623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:55.355679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.357410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5ece6f20-7036-499d-93c3-152f913acfb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:55.374173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:55.375430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:55.376882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:55.376882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:55.376882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:55.378896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 304/611] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:55.380429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b264e35-1753-4435-af42-645642a13285 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.380909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b264e35-1753-4435-af42-645642a13285 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.381402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b264e35-1753-4435-af42-645642a13285 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.387010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.387010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.387010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.387010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/793f922871f04a6480c571a073534d3f/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.387694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 305/612] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:55.389556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b264e35-1753-4435-af42-645642a13285 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:55.407234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=793f922871f04a6480c571a073534d3f, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.407882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:55.408699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9eabb-7f8e-4bab-91fb-caf8ccbad6cf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.501117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 308/613] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:55 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/793f922871f04a6480c571a073534d3f/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:14:55.508138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.508500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.508744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.511710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b264e35-1753-4435-af42-645642a13285 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:55.552451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:55.555803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:55.588386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.589472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.590765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.591272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.593422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:55.593946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.599881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.627004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 306/614] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:55.632955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:55.706557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 309/615] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:55.743793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:55.778061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e2c9-a4b8-489e-99eb-bbcac8370955 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:55.821780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:55.822423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:55.822697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:55.822965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:55.832241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:55.833690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:55.850986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:55.851961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:55.852885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 307/616] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:14:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:55.857620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:55.983878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:56.001144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd2a46-70ea-482c-a1fe-d78a93f1d928 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.014688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.038120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.038981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.039430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.039842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.040752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:56.041254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:56.042821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:56.065419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 310/617] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:14:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:56.070400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.211258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a71abf-bac2-43e6-9c85-69d746dc6150 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.251198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47075a3-e9d2-4fbf-947c-b30b99d8c1ba None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.267901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:56.277438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.285253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.285253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.285253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.305234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:56.317691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:56.318794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:56.319960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:56.323282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 308/618] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:56.328506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.332564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.334194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.334474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.334740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.363568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:56.365029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:56.367050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:56.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:56.398432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:56.399547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:56.399547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:56.399547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:56.402586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 311/619] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:14:56.406537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.493579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7d2a-ea6b-4d74-a8f0-8f7b551e2740 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.557782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.558406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.558684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.558957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.559672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:56.559974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:56.561356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262a673-225c-492e-ac8f-aa8f36c310ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:56.571223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 309/620] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:56.572494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e722d2-dfa1-4ae7-93da-4abdb19dc24b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.572776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e722d2-dfa1-4ae7-93da-4abdb19dc24b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.573050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e722d2-dfa1-4ae7-93da-4abdb19dc24b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.578575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 310/621] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:56.579972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e722d2-dfa1-4ae7-93da-4abdb19dc24b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.585954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038b6f3-4424-4204-8fe7-6b57fd8b24ba None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.625052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.626627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.627081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.627566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.628475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:56.628998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:56.630497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee835ef-546e-49b7-9243-11c93c77e3ea None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:56.649111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 312/622] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:56.649111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.649111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.649111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.663751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:56.665866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:56.667185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:56.721309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:56.729029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e722d2-dfa1-4ae7-93da-4abdb19dc24b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.747411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 313/623] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:56.747411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.798295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:56.799087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:56.799384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:56.799651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:56.820004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:56.821480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:56.847175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:56.848158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:56.849023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:56.849023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:56.849023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:56.849023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 311/624] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:56.849023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:56.964690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f1f3f-fa2c-47f3-9ea0-b2f6beed90d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.971125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61dbfa-b7ed-4231-9b1e-8f21fa4a7796 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:56.977080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.022792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.023894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.024180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.024450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.034969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.037768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.039718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.040336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.040609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.040875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.041519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:57.041818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.043189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:57.063843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:57.064784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:57.065646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 314/625] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:57.072530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 312/626] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:14:57.073416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.075079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.131712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.193173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9954d3f-851a-4650-bae1-5c5464eaf945 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.195390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d7d90-ad61-4ff6-a990-7cfbb1d41927 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.228793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.229395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.229676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.229845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.229967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.230449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.230625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:57.230733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.230930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.231028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.232389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.238826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.240290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ca9a3547-4792-45a5-8b9f-f310ffe40f60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:57.252439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:57.253422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:14:57.254445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:57.255719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 315/627] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:57.258183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 313/628] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:57.259095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.259397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5b80-8d51-4c08-9629-97a3aebf7f7c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.259665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5b80-8d51-4c08-9629-97a3aebf7f7c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.259935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5b80-8d51-4c08-9629-97a3aebf7f7c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.261548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 314/629] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:57.262231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5b80-8d51-4c08-9629-97a3aebf7f7c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.291592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.368595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7a113-ec00-4991-9c1e-86bc394458c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.370891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.377675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5b80-8d51-4c08-9629-97a3aebf7f7c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.404569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.405348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.405726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.406106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.413712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.414354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.414646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.414997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/d247a8be3add4804b86dcc0cd4cb72e3/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.417786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:57.423700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:57.424854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:57.425892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:57.427885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 316/630] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:57.428607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.440377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=d247a8be3add4804b86dcc0cd4cb72e3, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.442263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40977e06-9896-41be-b44f-615e89c96d17 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.529433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 315/631] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:14:57 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/d247a8be3add4804b86dcc0cd4cb72e3/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:14:57.531483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.531775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.532088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.533382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95a2948-2862-4c93-ac2c-b79fd79f6821 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.560289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:57.562108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:57.576474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.577097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.577417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.577696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.578795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:57.579080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.580837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.602377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 317/632] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:57.607886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.639776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.659892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 316/633] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:14:57.661071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.672205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.761529 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ab24b-06ce-4a0c-8b8a-547fb8afa52e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.791557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da12f9c-4e8d-40f9-95e0-0eafb120efd9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:57.814377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.814911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.815205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.815474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.816140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:57.816434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.817893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b344d9a-ced1-4c96-bad6-5d945078ea07 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.826828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 318/634] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:57.827997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fbb67-91a6-4b0b-a858-45e05e6bf463 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.828282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fbb67-91a6-4b0b-a858-45e05e6bf463 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.828551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fbb67-91a6-4b0b-a858-45e05e6bf463 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.834325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 319/635] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:57.835698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.835983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.836255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.838562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:57.839711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:57.840134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:57.840570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:57.855177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:14:57.856501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:14:57.914625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:57.916376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:57.925546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:57.932626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 320/636] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:14:57.940171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:57.974461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.048973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 317/637] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:58.048973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.074111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c2abb-b723-4356-b3b3-4274df12d5e6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.113208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.114569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.115262 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.115727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.130367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.132470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:58.153508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.154505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:58.155406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:58.155406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.155406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:58.155586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d13ee2-c140-4179-b1a4-d9211f2dc09a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.157636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 321/638] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.159380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.179833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.201682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.202462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.202840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.203429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.217111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:58.224001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.224921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:58.225751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:58.229743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 318/639] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.230491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.248454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.287688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ef2be-681e-4521-84b3-0563d0e55c85 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.326873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.327453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.327733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.327996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.340328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.342007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.347865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d251e-9261-43a8-9e64-ddf0394545f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:58.357172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.358082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:58.358946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:58.358946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.358946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:58.358946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 322/640] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.359681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.387658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.388267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.388543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.388811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.389442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:58.389736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.391101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.406597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 319/641] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:58.406597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.432643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.474178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97d8566-ae5b-46ff-ac99-742773048166 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.492130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.509836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ed726-ef5d-4c33-bc78-dba649ba2b8d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.510753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.511415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.511690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.511960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.523260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:14:58.528010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.529166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 323/642] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.530044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.552405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.553004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.553269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.553538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.554189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:58.554484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.555878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.564806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 320/643] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:58.565584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.647129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79160976-66c9-496d-8e32-67f727f19e18 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.686357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.686870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.687505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.687781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.688049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.688736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:58.689035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.690408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.699624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903ab0-0eaa-47fa-87cd-569d120c7632 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.702598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 324/644] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:58.703673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.737959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.738704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.739049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.739350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.747533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.748976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.749865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:58.761002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.765706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 321/645] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.767642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.817419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a9bb94-a862-4cab-8e8c-e908050f3335 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.855606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.856368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.856654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.857350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.858346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:58.858655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.861038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.871299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 325/646] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:58.872132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.881434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce63479-e31c-46d7-8d09-26fbbd228666 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:58.922931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:58.924275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:58.924773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:58.925252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:58.943270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:58.945926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:58.956173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:58.962908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:58.970240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:58.971178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:58.971178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:58.971178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:14:58.971178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 322/647] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:58.971178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:58.989723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:58.992487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a2c12d-2686-4a9d-8a02-a32821dfce95 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.031325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.032070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.032485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.032791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.039465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.040896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:14:59.056531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:59.057466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 326/648] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:59.058384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.087305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d5101-ffe2-4fac-880b-fa77fc9a701b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.123933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.124537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.124824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.125078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.126462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:59.127009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.128550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.146830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 323/649] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:59.147688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.182237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf0058-02fc-432f-8d09-1ad80f73f006 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.213155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.219003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.219909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.225036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.225036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.230750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.232219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:14:59.250063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:59.257230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:14:59.258325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:14:59.258325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:14:59.258325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:59.258325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 327/650] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:59.258325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.267851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.277951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66248e-43d3-4a75-802b-e04ca5bd0952 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.314919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.315571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.315829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.316098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.316989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:59.317106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.318377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.335311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 324/651] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:59.336692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.363130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ffe2e-beca-4798-aedd-6fa60e342a8d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.401253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.401860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.402132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.402412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.403103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:59.403430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.404843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.417449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 328/652] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:59.418156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.455892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b869618-da3c-4978-beb2-3248625c77f0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.484850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.490289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.511284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.511898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.512177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.512448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.531782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751603af-3648-40c4-84eb-5a9e081836a0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.575098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.576012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.576425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.576832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.577802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:59.578258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.580533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.592092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.594188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.602556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 329/653] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:59.603577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.696937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 325/654] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:14:59.697597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.704915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd062cc0-95c8-4786-a024-1a1755de45f2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.718760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.741042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:14:59.745732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.746355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.746631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.746901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.754038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.756727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:59.767737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:59.768749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 330/655] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:59.769744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.805532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efa032-c46f-41f4-a878-1b20258083ad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.838463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.839058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.839358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.839628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.840372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:14:59.840699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.842135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2dad8-31bf-4479-9251-8a6b1faca42b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.848429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 326/656] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:14:59.849578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6499-dc99-459c-8e2c-6d68703cc5c8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.849852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6499-dc99-459c-8e2c-6d68703cc5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.850120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6499-dc99-459c-8e2c-6d68703cc5c8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.851482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 327/657] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:14:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:14:59.852093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6499-dc99-459c-8e2c-6d68703cc5c8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.869610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba0827-431c-47fb-ba96-5f7e0db8e51c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:14:59.907620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:14:59.908284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:14:59.908559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:14:59.908828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:14:59.917139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:14:59.918627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:14:59.931338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:14:59.932493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 01 16:14:59.933624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:14:59.937391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 331/658] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:14:59.938382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:14:59.973122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6499-dc99-459c-8e2c-6d68703cc5c8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.012525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.012525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.012814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.013048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.025729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.027238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.037912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4d7f65aa-3837-43a3-a7b4-508f4dc3e053 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:00.045329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:00.049263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 328/659] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.050169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.054885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77855818-1937-4555-97b6-c28c03f7202a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.067211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:00.068427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:00.071913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.096190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.096801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.097080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.097355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.098086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:00.098393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.099847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.117357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 332/660] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Sep 1 16:14:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:00.118194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.151182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 329/661] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:00.151182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.224151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.229826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c4652-c361-48bf-982c-6e5e2100ee1e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.241552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.268314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a5e2e-489f-4d28-b8ca-d17265e701bf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.276101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.277152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.277764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.278798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.279349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:00.279755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.281296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.306846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 333/662] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:00.306846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.312756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.314058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.314340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.314615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.315295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:00.315656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.317097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.329732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 330/663] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:00.330689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.436917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec0b1b-7560-4ea1-a2aa-8525dab05501 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.438676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7767b32-acb2-47aa-9625-5bd52e7b3ca8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.463622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.475354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.475903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.476223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.476537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.502160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.502775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.503053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.503353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.548197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.554883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.556614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.596394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.597844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.690842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 331/664] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:15:00.698685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.722832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 334/665] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:15:00.724283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.727729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.799258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:00.809412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2fcfe-6101-48d6-b1a9-ec36e008a0f2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.840757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1cc60-dff9-4d7f-9c9c-c33ee23d3964 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:00.867203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.868004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.868286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.868557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.876783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:00.879054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:00.883961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:00.884593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:00.885096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:00.885318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:00.890699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:00.892135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:00.893015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 332/666] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:00.893514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.901747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:00.906775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:00.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:00.912424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:00.914696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 335/667] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:00.914696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:00.987568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.005556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7364f-e568-45e7-8fdd-11245ee3b95a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.028739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33553ee-5428-41fe-9bec-5442bbb71da7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.042580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.043268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.043570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.043818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.055813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.057274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.058023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.067458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.068093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.070259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.070259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:01.072941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:01.073830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:01.074726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:01.074726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.074726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:01.078536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 333/668] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:01.080063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.080207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.081714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:01.097741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:01.098721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:01.099655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:01.099655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.099655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:01.101016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 336/669] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:01.101920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.195257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d180d-763f-418e-b400-2e43ae82d6cc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.209935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d9ed49-7896-4e42-b3af-fa70135d388c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.229797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.230405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.230682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.230953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.231765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.231999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.233442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.236413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.247799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 334/670] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.247799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.264203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.265080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.265359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.265631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.266349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.267179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.268563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.276971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 337/671] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.280219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.335304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.366107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7bf0d-d893-418b-81df-4f6d8ba834a9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.397436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bd1ef-f937-417c-ab14-254e4bd81832 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.399699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.400306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.400588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.400861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.401502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.401801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.403190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.414364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 335/672] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.415215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.439541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.440507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.441006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.441459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.442469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.442930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.444591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.458009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 338/673] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.459218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.478085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.526129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb999bc-db7e-44c8-89a8-1b76b8ea62f5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.560118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.560719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.560992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.561253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.561962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=4a081c08049447949477e73e0891f1e8) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.562280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.564379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.568855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132660-b2f5-46df-857b-e2337aeebff5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.570391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.579503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 336/674] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/role_assignments?user.id=4a081c08049447949477e73e0891f1e8&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.579503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.620356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.620997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.621274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.621544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.635048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.636664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:01.655773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:01.657462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:01.658380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:01.658380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:01.658380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:01.662913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 339/675] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:01.663915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.721759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c70d06-db47-45eb-9635-90143eb04542 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.756306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.777535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.779422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.780550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.781379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.785162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c95848-b1ff-40f1-bffa-2c701f899685 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.789085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.791527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:01.805362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:01.806306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 337/676] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:01.807783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.823965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.824625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.824900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.825168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.836306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.836811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:01.851414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:01.852524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:01.853586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:01.858711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 340/677] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:01.859598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.885186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:01.928466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db228edb-1b1d-4e76-9618-57f8e3c2c9fe None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:01.962309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:01.962946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:01.963256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:01.963534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:01.964177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:01.964475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:01.965853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:01.978369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 338/678] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:01.979211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:01.990845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb1a88-2c6c-4bd9-9765-53d31f30f3be None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.025149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.026390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.027009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.027725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.028047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.028668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:02.028972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.030412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb2bff-6641-4fe4-9edf-d3c105179302 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 341/679] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca26ca-e7d7-4c38-80ae-9d697270ebfb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca26ca-e7d7-4c38-80ae-9d697270ebfb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca26ca-e7d7-4c38-80ae-9d697270ebfb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 342/680] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:02.049124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca26ca-e7d7-4c38-80ae-9d697270ebfb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.099988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30daa-707c-4038-b974-85d1fec92e33 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.131472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.138392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.139035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.139351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.139674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.140423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:02.140725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.142762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.155098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 339/681] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:02.160168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.163576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca26ca-e7d7-4c38-80ae-9d697270ebfb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.203237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.204530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.204945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.205350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.213186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.214777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8bb7e0b5-217d-4647-bb22-b8c5e537be2f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:02.227326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:02.228552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:02.229677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:02.230211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 343/682] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:02.231130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.231435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.231709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.252899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:02.254630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:02.275657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.278980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f27c1b-c487-457d-938b-d8074d348f3f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.314642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.315284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.315602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.315880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.316615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=793f922871f04a6480c571a073534d3f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:02.316929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.318939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.335188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 340/683] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/role_assignments?user.id=793f922871f04a6480c571a073534d3f&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:02.335188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.355599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 344/684] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:02.356811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.492722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.521907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75271cf-9826-461c-b7de-d5cc730ef303 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.548406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8122bb1c-40fc-4993-9821-5d84b5056e71 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.572836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.573857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.574543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.574972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.575895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=e346b0a022ea4218a4bd10c7e6fe6d91) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:02.576350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.581146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.589576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 341/685] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/role_assignments?user.id=e346b0a022ea4218a4bd10c7e6fe6d91&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:02.607866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.609244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.609244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.609443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.610179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:02.610530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.612065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.631011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.642878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 345/686] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:02.649141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:02.682161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.772933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6518c25d-0fc4-469d-bc99-8f751f3b57e0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.799109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:02.813145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.814024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.814380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.814650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.849120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7a414-168b-4aac-a157-897ea747042e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:02.886603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:02.887657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:02.888167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:02.888638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:02.889857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:02.892244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:02.902060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:02.908774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:02.909708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:02.910534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:02.913365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 342/687] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:02.921359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.024930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 346/688] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:15:03.032495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.039434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.054547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87585abf-a704-46be-ac25-0e21cc66db03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.059804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.093805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.095998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.096560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.097138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.098200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:03.098712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.100739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.114827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 343/689] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:03.117490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.146209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949aef87-dbf7-48bf-a2bf-77eef6ffebbf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.184563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.185195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.185469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.185738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.196783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.198511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:03.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:03.215794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:03.217366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:03.217366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:03.217366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:03.217366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 347/690] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:03.222382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.237087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9000278e-c242-4883-8d38-099c8f59bb1d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.275488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.276278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.276672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.277076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.290059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.291920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:03.308897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:03.309840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:03.310865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:03.310865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:03.310865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:03.314984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 344/691] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:03.316984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.319761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.323344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.349290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f6adc-9a84-4a0c-9826-fdfb48541b76 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.386620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.387433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.387805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.388172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.388915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:03.389312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.390874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.412744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 348/692] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:03.418251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.431986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f291184-84a3-4540-9cea-f580a62228f9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.468922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.470562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.471045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.471620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.472492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:03.473012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.474714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.495295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 345/693] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:03.497370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.547237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104c7ae-833f-409e-8e87-9ed44c7e184e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.581699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.581699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.581699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.581699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.585312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.586488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.588444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.597265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:03.601440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:03.604662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:03.605589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 349/694] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:03.608461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.705508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae216e6-08a0-4ddc-80ea-7ebb53f1cf23 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.715309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc271604-7d59-4f6d-b206-51eaa3cbaa43 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.752427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.753442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.754051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.754333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.755323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:03.755632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.757793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e20b6-5580-40ee-8d37-551df7c93c4e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.761417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.762057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.762335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.762608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.773221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.773387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 350/695] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:03.774651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.779479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23bc0-8621-4e54-81bf-758027559024 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.780411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23bc0-8621-4e54-81bf-758027559024 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.780690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c23bc0-8621-4e54-81bf-758027559024 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.782849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 351/696] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:03.783852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c23bc0-8621-4e54-81bf-758027559024 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-192266a3-523f-414b-b204-fbb0bee615d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:03.796653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:03.797689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:03.798979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 346/697] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:03.800236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.800368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.801438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.823571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:03.824850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:03.851565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.900995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 347/698] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:03.901889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:03.908784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c23bc0-8621-4e54-81bf-758027559024 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:03.968122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:03.975493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=d247a8be3add4804b86dcc0cd4cb72e3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:03.985282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:03.995980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 352/699] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/role_assignments?user.id=d247a8be3add4804b86dcc0cd4cb72e3&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:04.033594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700cc037-95dd-4852-9632-1df6f620eb90 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.080410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.081228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.081393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.081594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.082480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:04.082788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.084333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.104969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 348/700] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:04.112304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.239252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4563398-a4e6-4b51-aea1-b4b0262cf898 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.258108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:04.278633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.279304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.279585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.279854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.338953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.356445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.357837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.456164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944d7b-9b1e-443b-b870-5572d86c54a2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.465958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 353/701] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:15:04.468396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:04.472128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.490771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:04.495715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.496350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.496630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.496894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.497470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.498850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:04.511973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:04.513019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:04.513924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 349/702] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:04.519759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.584954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962421f9-dfb4-4549-b359-38c3ffb9b00d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.623584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.624537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.625018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.625514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.633260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cb101-50f8-496f-a2b3-73e366be3ab6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.637681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.639424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:04.656182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:04.657818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:04.658769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:04.658769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:04.658769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:04.660941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 354/703] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:04.666833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.675932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.677206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.678258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.678736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.680161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:04.680669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.683080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.690919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 350/704] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:04.703312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.767618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:04.779090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:04.799558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f02fc4-2b10-4a58-b4a5-ed7de9589534 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.839106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.840040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.840612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.841036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.842320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:04.843042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.845643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f1244-74f6-4f99-947d-31e47d94f44e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.860746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 355/705] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:04.862450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a876-819d-47a5-bfc0-1d41c0f5abca None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.863335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a876-819d-47a5-bfc0-1d41c0f5abca None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.863763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8a876-819d-47a5-bfc0-1d41c0f5abca None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.865538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 356/706] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:04.866377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8a876-819d-47a5-bfc0-1d41c0f5abca None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:04.886894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c179674f-8084-4294-ad3c-d3c033f4c6ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:04.925305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48e277-afed-440d-a676-1b762239176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:04.925887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e277-afed-440d-a676-1b762239176c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.926216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e277-afed-440d-a676-1b762239176c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.926488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48e277-afed-440d-a676-1b762239176c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.927447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e277-afed-440d-a676-1b762239176c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:04.927801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e277-afed-440d-a676-1b762239176c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:04.929246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48e277-afed-440d-a676-1b762239176c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:04.943213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 351/707] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:04.943213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:04.943213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:04.943213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:04.957955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:04.967247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:05.004202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8a876-819d-47a5-bfc0-1d41c0f5abca None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.043609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:05.049730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.050456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.054327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.054327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.059924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:05.062024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:05.076856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:05.077809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:05.079692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 357/708] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:05.088199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.098854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 352/709] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:05.221713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdde029-f58a-4428-8fd3-17dd8bac812c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.269986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.270662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.270949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.271242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.271953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:05.272255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:05.273678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ee789-d14b-49d0-aa12-3492710998a2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:05.290431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 358/710] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:05.296500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.346084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:05.376750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebca7e-14f1-4da4-a2f9-4a83058b86f9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.377378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebca7e-14f1-4da4-a2f9-4a83058b86f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.377875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebca7e-14f1-4da4-a2f9-4a83058b86f9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.379684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 359/711] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:05.384793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.385307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.385790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.408595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:05.410107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:05.460800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63a6192-3f94-47aa-9f55-e7aee0261c45 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.466168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:05.488873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 360/712] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:15:05.490032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.517772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.518943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.520367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.521619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.522796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:05.524304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:05.526388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:05.538255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 353/713] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:05.656430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db9155-99ad-43eb-903c-65fbe8ebcc42 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.703495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.704181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.704454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.704810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.705403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:05.706772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:05.717694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:05.718749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:05.720166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 361/714] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:05.727378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.826370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.853827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:05.862695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1ae9-d78e-4b68-97db-50a4cb1f08d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.884860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:05.928886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.929504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.929790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.930064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:05.930873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:05.931091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:05.932562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:05.945396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 354/715] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:05.950674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2574a1-345a-46f0-9bff-07eef9bbcba3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:05.951072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:05.996800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:05.997449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:05.998574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:05.998831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.023482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:06.025045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:06.051723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:06.053021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:06.053923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:06.053923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:06.053923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:06.053923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 362/716] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:06.060757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:06.148237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ea6dc-a19f-40d1-b636-4ba2829c4a66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:06.213795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:06.214412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:06.214690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:06.214959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.215623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:06.215919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:06.217311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:06.230917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 355/717] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:06.254301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:06.266011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db406e0-9beb-438b-8d8f-749320a24d60 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:06.310632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:06.311288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:06.311571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:06.311918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.312592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:06.312897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:06.315077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e536fa-5825-4c92-8831-1664f71cd3e7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:06.329387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 363/718] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:06.336442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:06.376010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:06.463467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63289e9-195f-437a-8fbf-c02383987b09 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:06.500430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:06.501497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:06.502012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:06.502477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.503424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:06.505206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fafb8c-59b7-4035-b3e8-4e0cfe037ab4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:06.611115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 356/719] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:15:06 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:06.716801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c38cf-4890-45dd-9647-c2b44d7b9948 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:06.717107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c38cf-4890-45dd-9647-c2b44d7b9948 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:06.717273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c38cf-4890-45dd-9647-c2b44d7b9948 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.718975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 364/720] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:06.730110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0329-151f-46cd-8d1c-dea87fe3b27b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:06.730731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0329-151f-46cd-8d1c-dea87fe3b27b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:06.731247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d0329-151f-46cd-8d1c-dea87fe3b27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:06.759885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3d1d0329-151f-46cd-8d1c-dea87fe3b27b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:06.761631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d1d0329-151f-46cd-8d1c-dea87fe3b27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:06.843462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 357/721] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:07.128738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c38cf-4890-45dd-9647-c2b44d7b9948 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:07.252799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c38cf-4890-45dd-9647-c2b44d7b9948 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:07.301343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:07.302598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:07.303085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:07.303570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:07.318507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef1dc-bb97-4dde-a972-3d8849ad0e96 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:07.319020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef1dc-bb97-4dde-a972-3d8849ad0e96 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:07.319020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ef1dc-bb97-4dde-a972-3d8849ad0e96 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:07.320626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 358/722] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:07.322466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:07.325541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:07.325817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:07.326087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:07.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:07.330445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:07.331334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:07.333950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 365/723] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:07.339361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:07.366293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:07.372168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:07.373409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:07.393108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:07.485874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 359/724] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 01 16:15:07.495608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ded0a-b0da-41df-8dc1-3701a747a8e5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:07.599903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:07.600813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:07.601232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:07.601686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:07.602671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:07.603123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:07.604658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:07.618770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 366/725] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:07.624303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:07.736006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e78d73-d250-4b28-9630-6dc0eb9f2fdf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:07.770777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:07.771438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:07.771716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:07.771980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:07.780915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:07.787843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:07.789365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:07.806071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:07.807168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:07.808080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:07.808080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:07.808080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:07.811609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 360/726] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:07.820404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:07.885005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.016952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabf4a6-3c6b-4965-aec9-5a8668376872 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.023127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:08.051229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff12fb-f901-4ddf-b5c6-59d19dd48156 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.060034 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.060653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.060932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.061205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.061989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:08.063575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eea3913-db84-4eee-b063-8dc519da87a0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:08.088557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.089509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.089509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.089710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.090381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:08.090682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:08.092080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e62f-3abe-46ae-90bb-2d3a9dd00f3d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:08.105122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 367/727] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:08.109575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8e77-1605-4608-a737-66add1105059 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.109575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8e77-1605-4608-a737-66add1105059 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.109575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d8e77-1605-4608-a737-66add1105059 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.109575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 368/728] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:08.111115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d8e77-1605-4608-a737-66add1105059 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.183452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 361/729] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:07 2022] POST /identity/v3/services => generated 251 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:15:08.185085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.185374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.185928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.227738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:08.228969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:08.286311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d8e77-1605-4608-a737-66add1105059 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.329930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.330586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.330864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.331135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.333948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:08.342963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:08.344679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:08.349090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:08.362821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:08.364023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:08.365783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:08.368788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 369/730] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:08.369896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.385499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 362/731] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:08.490446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdc1da-22b7-4677-b9e0-591daef02da3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.526766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.527404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.527735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.528013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.528752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:08.529052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:08.530492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ad81a-13d3-42f1-8f90-108f8b34db6d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:08.548834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 370/732] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:08.554638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.632854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379e518-a1e9-4d68-9404-4fb06f440dfc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.633139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379e518-a1e9-4d68-9404-4fb06f440dfc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.633411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379e518-a1e9-4d68-9404-4fb06f440dfc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.634865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 371/733] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:08.639898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.640232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.640512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.660434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:08.661693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:08.664387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9c69d-0bd5-47e5-99bf-dfc6bca45985 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.701311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.702291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.702790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.703292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.716812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:08.739911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 372/734] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 01 16:15:08.740764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.795499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:08.827478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:08.829682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:08.850249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b0b8c-88b2-461a-ba34-d5ae5f781680 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:08.886565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:08.887229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:08.887509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:08.887784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:08.899776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:08.905198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:08.906126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:08.906982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 373/735] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:08.914487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.979715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 363/736] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 01 16:15:08.991773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:08.997298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.095525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bf138-b56b-406f-bae3-abbe92dcd173 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.131388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.132045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.132323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.132642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.133319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:09.133625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.135754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.139798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.147369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 374/737] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:09.148549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.168077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad78c5-4bc1-4c63-b80d-d295c7cd3312 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.213526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.214490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.214948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.215437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.228779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.230502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:09.248615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:09.249526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:09.250413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:09.250413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:09.250413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:09.250413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 364/738] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:09.250635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.284499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1b214-1310-4177-b5b4-4cb683d2b68b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.287652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.326065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.326743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.327022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.327346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.327992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.329541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.370253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305fe24-7a12-4c0c-8cea-853bc7390634 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.398010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 375/739] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:09 2022] POST /identity/v3/services => generated 240 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:09.404692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.405730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.411255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.416869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.416869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.416869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.425278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.427101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-45d1a259-e413-45c1-af83-359223b40bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:09.443224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:09.449398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 365/740] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e31cd8-0786-40dd-8017-9c8bb68c07a0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e31cd8-0786-40dd-8017-9c8bb68c07a0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e31cd8-0786-40dd-8017-9c8bb68c07a0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.450521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 366/741] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:09.455194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.455194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.455194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.476229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:09.477655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:09.570413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 367/742] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:09.571932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.575143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3b02d0-72ce-4d05-af98-06abc6021fbe None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.634078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.635131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.635706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.636157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.637238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:09.637729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.640131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.664264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 376/743] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:09.671420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.706630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.736263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5506a9-0377-4110-bc75-a381006bb47c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.756620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:09.790876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.791604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.791908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.792839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.793521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:09.793825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.795442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.797567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688f8f2-fe46-4b68-bd5a-d043410b5b45 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:09.823092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 368/744] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:09.827643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.854994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:09.856019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:09.856504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:09.857005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:09.870803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:09.872607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:09.888304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:09.889216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:09.890110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:09.898325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 377/745] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:09.904206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:09.973701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960c4fe-f4a5-4799-95d4-b645905fa5bc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.011048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.018324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.019338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.019827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.020395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.026599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74516-959e-45fc-8ff9-67c640ed8fbf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.029329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.031029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:10.043965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:10.044958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:10.045907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:10.053579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 369/746] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:10.054764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.062197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.062832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.063110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.064104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.074924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:10.080851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:10.081891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:10.082717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 378/747] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:10.083504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.095413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.174476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477110d8-9126-453c-8da8-c95d7ced7f80 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.187827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9d61f-e01d-4f11-8ebb-960f1d2dbc08 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.212771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.213399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.213713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.214059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.214876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:10.215216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.216933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.223882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.224156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.224536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.225271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:10.225575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.227095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.230262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 370/748] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:10.231200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.260063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 379/749] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:10.261165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.264256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.380169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6049b-8edd-449a-9cde-0c8c8833ed34 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.416005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.416714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.416995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.417265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.418110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=d247a8be3add4804b86dcc0cd4cb72e3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:10.418395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.419843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.426931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 380/750] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/role_assignments?user.id=d247a8be3add4804b86dcc0cd4cb72e3&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:10.428029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.464814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.478022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7bca7-a6e8-468a-bc21-fe661b5873a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.503872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.541544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e42ad-824f-4beb-beba-347946a57b24 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.555644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.556580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.557101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.557551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.558480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:10.558928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.560597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.580507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.580897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 371/751] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:10.581738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.581738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.581738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.584801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.593559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.601171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:10.611166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:10.612510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:10.613737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:10.613737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:10.613737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:10.613737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 381/752] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:10.618268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.728705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b860bb0-d701-4e09-8728-a55ab1151d83 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.749028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.763803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc13e08-1630-4615-af3f-aa583db10859 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:10.779194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.779848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.780548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.781201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.782977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:10.783477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.785440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc1986-035c-496b-b7b4-80ce2b7836ad None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.804540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 382/753] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:10.805879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dad10-59a4-43dd-a36a-dbb8149a4e35 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.806168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dad10-59a4-43dd-a36a-dbb8149a4e35 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.806442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066dad10-59a4-43dd-a36a-dbb8149a4e35 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.807579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:10.811767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 383/754] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:10.811767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dad10-59a4-43dd-a36a-dbb8149a4e35 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.811950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.812444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.812444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.824427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:10.826245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-13099831-689d-4089-8dd4-750ad5eda662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:10.841881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:10.843880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 372/755] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:10.846257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:10.859879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:10.866247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:10.867503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:10.939672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 373/756] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:10.943445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:10.965821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066dad10-59a4-43dd-a36a-dbb8149a4e35 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.020748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.021400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.021679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.021951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.029678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.031295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:11.049602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:11.050528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:11.051506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:11.056734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 384/757] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:11.060721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.132711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376129d-18fa-4ab8-ac18-1d954ed34087 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.148982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.197201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.198207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.198689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.199192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.200129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:11.200629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.202314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.205542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.220733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 374/758] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:11.225940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.264711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ed6b-cbf5-46dc-a196-b8a5566de380 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.304647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.305589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.306124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.306710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.307634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:11.308145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.310030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.323799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 385/759] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:11.325027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.361214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d9c577-daec-4d4c-b276-fe33aae0325e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.397929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d091d-c555-4293-b328-fcee334c135a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.398918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d091d-c555-4293-b328-fcee334c135a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.399420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d091d-c555-4293-b328-fcee334c135a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.399912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d091d-c555-4293-b328-fcee334c135a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.408103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d091d-c555-4293-b328-fcee334c135a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.409839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d091d-c555-4293-b328-fcee334c135a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-010d091d-c555-4293-b328-fcee334c135a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:11.421827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:11.422853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:11.424268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:11.426611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 375/760] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:11.427938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d091d-c555-4293-b328-fcee334c135a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.452988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.458637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836190bf-6a7e-4a1e-8867-4a163543a15c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.495044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.501112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.502185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.502687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.503200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.504027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.505658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:11.515220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:11.516245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:11.517139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:11.520228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 386/761] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:11.521308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.548399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d091d-c555-4293-b328-fcee334c135a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.596684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.596684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.596684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.596684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.598826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=68764b02b584408090a0c12a06b50445, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.600560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.632954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8d0b7-043d-4e3a-86bf-dc6a83a1cbb1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.668638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.669372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.669653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.669926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.670718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:11.671023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.672491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997e87b-0384-4c74-a43e-8066af09d4b2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.691189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 376/762] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:11 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:11.691189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.701918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 387/763] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:11.708729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f1118-f835-47a1-9149-acd1d607a178 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.709024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f1118-f835-47a1-9149-acd1d607a178 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.709277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f1118-f835-47a1-9149-acd1d607a178 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.710942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 388/764] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:11.711725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f1118-f835-47a1-9149-acd1d607a178 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:11.743597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.808047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43594aa-65ab-4863-8e4c-b52af0ff1e2f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.840420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:11.856480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:11.857516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.857974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.858413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.859272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:11.859753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:11.860942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f1118-f835-47a1-9149-acd1d607a178 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:11.861833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b79c2-7af7-4e2e-a9c9-cb878d80a97d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:11.869449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 377/765] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:11.871186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:11.871767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:11.872282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:11.893630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:11.895289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:11.980292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 378/766] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:11.981189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.026210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.026857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.027135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.027445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.102864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.104402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.122611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad1dd-71f2-4a31-9ef8-31dcb23e5d82 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.135500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.144237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.167919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.168818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.171292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.171752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.172603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:12.173229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.176353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.186495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 379/767] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:12.193252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.280609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 389/768] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:12.286587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.358054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f05f6-28d0-432b-a6af-c46673a4340e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.394545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.395100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.395417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.395673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.396376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] RBAC: Adding query filter params (interface=public, service_id=5ecee1296d9e42f7b3521686c451060d, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:12.396676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.398046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.406121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 380/769] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/endpoints?service_id=5ecee1296d9e42f7b3521686c451060d&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:12.406949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.418986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9143a0-b1a5-4a61-a3af-30fd8bb990fd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.453479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.455056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.455749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.456033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.456310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.459477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.468178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.472164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:12.484816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:12.485879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:12.486773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:12.486773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:12.486773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:12.492048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 390/770] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:12.492960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.534553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a5c2e0-2437-4ef3-bff7-c70c8ea83fb6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.584326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.584823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.585104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.585379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.586133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.588021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:12.600521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:12.601536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:12.602375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 381/771] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:12.603208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.613136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da56b8d-3a5e-416f-b51c-f1279971f924 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.651388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.652346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.652828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.653299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.654113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.655735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.661958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 391/772] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:12.663025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.715259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.793119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2fef2-92da-4c79-a030-4e965425b157 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.794808 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:12.838673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6cff6-bd28-4965-ae14-c3c19a4046e9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:12.839568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.840196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.840680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.840957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.842536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:12.842848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.846273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9f8e9-ba0f-4eed-bb62-51ca5a91f455 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.860347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 382/773] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:12.861594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c22e-a699-44e4-b3d7-cd299ea61c41 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.861886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c22e-a699-44e4-b3d7-cd299ea61c41 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.862158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25c22e-a699-44e4-b3d7-cd299ea61c41 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.863392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 383/774] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:12.865712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25c22e-a699-44e4-b3d7-cd299ea61c41 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:12.884770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:12.884770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.884770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.884770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.885406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] RBAC: Adding query filter params (name=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:12.886099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:12.887030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363a852-eeaf-4328-8d61-60c9d9c4d126 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:12.896934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 392/775] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:12.896934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:12.896934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:12.896934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:12.918379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:12.919628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:13.002229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25c22e-a699-44e4-b3d7-cd299ea61c41 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.007464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 393/776] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:13.008437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.043096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:13.113960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.114604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.114884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.115176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.122564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763b689-df7f-4dae-9bb8-b410fa1205b0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.132206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.133741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:13.157536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:13.158462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:13.159376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 384/777] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:13.166769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.166990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.167608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.167890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.168163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.168858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] RBAC: Adding query filter params (type=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:13.169106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.170584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.179380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 394/778] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.191973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.221644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:13.353591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:13.356376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b960a31e-35e7-4eb9-8214-8463a0be4386 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.369248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19207c4c-4d44-4e5f-b94d-fcba50599398 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.421655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.422189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.422465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.422776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.423584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:13.423902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.425350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.426961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.427965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.428351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.428835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.429641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] RBAC: Adding query filter params (interface=public, service_id=80165550fb3744358e56dacf03d0990e, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:13.430020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.431484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.439956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 385/779] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.441194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.443230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 395/780] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/endpoints?service_id=80165550fb3744358e56dacf03d0990e&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.445113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.545574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:13.554820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1806e7a-afaf-4c6c-b572-e8086b7b9b6a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.570595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15bc6d-b5d5-41b5-850a-dcbdf0b8c9a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.592575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.593281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.593561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.593830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.594441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.595906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.607180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 396/781] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.607180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.633734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.635238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.637462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.639271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.679938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:13.683466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:13.689856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:13.690953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:13.691832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:13.699486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 386/782] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:13.706834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.810000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd270b3e-c192-4ac2-87d2-4896ddefd083 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.832430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718ebc3-4fb0-41a3-bfa6-474eee8e6a70 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:13.851770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.852532 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.852912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.853283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.854132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=711f1b63cfa1461da6b5b9bb286c6f95) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:13.854540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.856210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d6ede8-f512-4a03-a4ef-7be506d2051a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.867194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 397/783] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/role_assignments?user.id=711f1b63cfa1461da6b5b9bb286c6f95&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.867194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f185ae-8bf2-408d-8dae-3d9054c73c86 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.867194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f185ae-8bf2-408d-8dae-3d9054c73c86 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.867194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f185ae-8bf2-408d-8dae-3d9054c73c86 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.867194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 398/784] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:13.873774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.874183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.874565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.887994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:13.889592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:13.890023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:13.890773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:13.891681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:13.892132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:13.893845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:13.895062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:13.896696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:13.912167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 387/785] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:13.921721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:13.930642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.040125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 399/786] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:14.087331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9c799-657a-4fc1-83bd-152e85c90253 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.123121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.123782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.124051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.124369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.135447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.136903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:14.152775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:14.153755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:14.154626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:14.154626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:14.154626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:14.158651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 388/787] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:14.160049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.179529 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.276494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f766b7d-a934-490f-88cb-54a7fe4427f4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.313020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.313705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.313978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.314250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.314893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:14.315213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.316579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.328543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 400/788] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:14.334375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.379527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.423538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.463750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.475354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268db8-6248-4409-a2df-ced9d168bfd2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.489445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676a20a-661e-47b3-99e9-181bd0c931f6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.511214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.511783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.512116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.512389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.520784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.522295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:14.534509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:14.535487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:14.536415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:14.540348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 389/789] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:14.546376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.555064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.556903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.557376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.590203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:14.596117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:14.597129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:14.597996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:14.603501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 401/790] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:14.611599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.647704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b9565c-199a-4e9f-b1e5-0bfbc94e72ce None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.685495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.686102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.686377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.686648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.687510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:14.687810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.690027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.711688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.713472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 390/791] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:14.714773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.798356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7138f11-46f5-4941-8de1-ddfa19624e9b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.821675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed335bdb-45c9-49ed-a40a-aa2f72fa43f9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:14.857315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.857952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.858231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.858499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.870488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.871985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:14.887594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:14.888555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:14.889445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:14.889445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:14.889445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:14.893693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 391/792] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:14.895011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:14.908558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:14.909514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:14.909999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:14.910471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:14.911409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:14.911938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:14.913540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:14.920305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.935575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 402/793] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:14.935772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:14.942354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.040459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e0a6c-a447-4ee4-80d9-2628bbf1f8f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.086205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.086205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.086205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.086205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.086614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:15.086614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.086614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eee854e-a16b-495d-8a9b-2b4beb807654 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.089038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0add2b87-cae3-4de2-925d-a9085e596790 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.101149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 392/794] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:15.102679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42105b-28b8-4848-918f-8a339a5a961f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.103172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42105b-28b8-4848-918f-8a339a5a961f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.103874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a42105b-28b8-4848-918f-8a339a5a961f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.105605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 393/795] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:15.106505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42105b-28b8-4848-918f-8a339a5a961f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.147188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.147810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.148082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.148345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.163343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.165587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9d0feeac-9f7c-48f9-809c-52511f1377cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:15.186942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:15.188105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:15.189785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:15.189785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:15.189785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:15.196080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 403/796] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:15.197702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.198310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.198589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.213841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.218789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:15.220874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:15.231626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a42105b-28b8-4848-918f-8a339a5a961f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.279028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.285834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.286742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.287267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.287828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.297587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.300647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.312153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 404/797] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:15.313159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:15.315057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:15.316145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:15.317280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:15.319684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 394/798] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:15.328327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.492691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946af7e-b628-4af0-8b1c-a3b66c58d906 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.529124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.529895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.530179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.530447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.531133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:15.531455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.533626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.537086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1f335-6189-46ab-8d6e-2f7c1e5bfc6f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.546613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 395/799] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:15.554248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.568867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.597479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.598254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.598352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.599177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.599284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:15.601577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.603645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.630753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 405/800] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:15.638208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.689104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.694702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d2475-b663-4e34-b753-315fa2cd1e5f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.731559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.732232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.732512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.732786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/d247a8be3add4804b86dcc0cd4cb72e3/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.735713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=d247a8be3add4804b86dcc0cd4cb72e3, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.737375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.751787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788070d7-8ca6-466f-8dd6-e60b66a4d067 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.787745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.788387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.788666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.788935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.796823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.798945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:15.810126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:15.811137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:15.812160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:15.815232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 406/801] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:15.816337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.839367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 396/802] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:15 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/d247a8be3add4804b86dcc0cd4cb72e3/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:15.840822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:15.930406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.933516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783c33-444a-4ab4-8b11-02e933a12fd1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.949997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:15.976416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:15.977172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.977526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.977878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:15.978553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:15.980230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-beb67b66-b46a-49df-a508-4190d7362009 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:15.990728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:15.992074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:15.992898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 407/803] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:15.993799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91b568-64ee-4c1a-b26a-58dee91fe0ab None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:15.996860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be32b-7996-42b0-89e8-0da977a9e0c3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:15.999054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91b568-64ee-4c1a-b26a-58dee91fe0ab None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:15.999423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91b568-64ee-4c1a-b26a-58dee91fe0ab None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.001743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 408/804] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:16.002688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91b568-64ee-4c1a-b26a-58dee91fe0ab None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.041697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.054515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b7c36-432b-4959-b2a9-fbfbe8c04007 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.069545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 397/805] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:16.071383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.071672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.071944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.094545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:16.095899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:16.150696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91b568-64ee-4c1a-b26a-58dee91fe0ab None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.166092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 398/806] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:16.166776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.200809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.201811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.202269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.204770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.205599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:16.207100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.208737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.214670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 409/807] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:16.221000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.259733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:16.273035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:16.308463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525999df-2fa5-49d4-9ce3-cada0d585eb5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.347188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.348027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.348380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.348730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.375269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019ba0-af5f-4891-9585-833871c6d349 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.412407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.413019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.413299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.413573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.414208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:16.414511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.415913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c21011-ddfe-4c06-9943-26db347e14fb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.418675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.420251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7eeaf-4dfe-4b6f-855f-8cc373c5cc87 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.421444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 410/808] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:16.422639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ae47-3649-41f5-8372-f59dc58c2e1c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.423542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ae47-3649-41f5-8372-f59dc58c2e1c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.423818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9ae47-3649-41f5-8372-f59dc58c2e1c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.425338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 411/809] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:16.427067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9ae47-3649-41f5-8372-f59dc58c2e1c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.491627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:16.524326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 399/810] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:16.525689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.525976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.526298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.531531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:16.542751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9ae47-3649-41f5-8372-f59dc58c2e1c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.545762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:16.546476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:16.580914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.581508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.581791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.582064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.582736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.584133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.618933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 400/811] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:16.620005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.707874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 412/812] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:16.710427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.711811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:16.749581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354cd4d-11dd-496b-95a8-c1c9977a993b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.841228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.841794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.842069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.842335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.867947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.869420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:16.896651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:16.897641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:16.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:16.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:16.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:16.900425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 401/813] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:16.901479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.916710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12abaa-b4ea-4691-9152-ca9084a59f48 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:16.958670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:16.959357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:16.959632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:16.959904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] PATH_INFO: `/v3/services/network` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:16.960494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:16.961860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:16.973611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:16.974830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:16.975722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 413/814] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/services/network => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:16.976547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:16.999103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.031561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe0aea-29d3-405f-8931-64e7e5ac79d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.069253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.079741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.080601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.080878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.081151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.097416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:17.104250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:17.105791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:17.107673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:17.108959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c9d2ff-7d5e-45e1-9ffc-a5f79a778320 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.112677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 402/815] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:17.114949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.147492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.148296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.148675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.149048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.149800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.150200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.151707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.164980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 414/816] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.165948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.232358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2392d-13e9-4f4e-9757-699647b0235c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.243142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.277373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ccd8e-d644-4bcf-bda4-e96b775e727c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.278143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.278763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.279043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.279351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.280013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.280397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.282086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.287804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 403/817] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.288656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.318296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.319330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.319833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.320222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.320994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.321399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.322859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.336948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 415/818] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.336948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.361585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.400533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5dd3c-802c-4430-9eb6-df54cb23a5fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.447334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.447981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.448262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.448604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.456982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884035-520e-4752-908b-0ef13a9cdd92 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.458166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.460975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:17.476056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:17.477162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:17.478356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:17.482107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 404/819] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:17.483407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.508063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.509238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.509680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.510097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.510873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] RBAC: Adding query filter params (type=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.511820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.513714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.515893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.522357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 416/820] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.524025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.605190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840018f5-e948-4374-b364-c325eb967c77 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.624500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.644475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.645054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.645331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.645599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.659205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.660209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca35fe-822a-40ef-8995-8c1c82bcee22 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.660666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:17.677225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:17.678661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:17.679843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:17.679843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:17.679843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:17.679843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 405/821] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:17.679843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.711118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.711748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.712026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.712297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.712999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.713460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.715041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.728889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 417/822] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.729716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.805901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:17.809774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd035f43-1ad4-4312-bbaa-a4ec21b4afe0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.848244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.848952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.849301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.849736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.850584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.852107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.860947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d947bd-620e-4a7e-9004-bf51a1b96c06 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:17.899231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:17.899774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:17.900051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:17.900377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:17.901029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:17.901339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:17.902750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:17.919320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 418/823] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:17.919320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:17.959254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 406/824] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:17 2022] POST /identity/v3/endpoints => generated 337 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:17.960586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.011576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.045103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f799e-5f5f-46fe-8208-10a2ef969d14 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.071650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.097503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1cf1-0c7d-4ba7-a8c3-cec15c3906e4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.102925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.103896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.104342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.104779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.105688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:18.106841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.108907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a769a03c-5edb-47b2-8ee3-955bdd777eb2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.118692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 419/825] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:18.120274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e88d15-78bc-4236-81b5-5c4500d0ae81 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.120742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e88d15-78bc-4236-81b5-5c4500d0ae81 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.121930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e88d15-78bc-4236-81b5-5c4500d0ae81 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.123669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 420/826] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:18.132922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.133250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.133476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.150181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.151114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.151559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.151971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.156122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:18.156887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:18.164495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.166224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:18.180185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:18.181361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:18.182500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:18.184573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 407/827] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:18.191301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.262435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 421/828] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:18.299931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.321170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08070a60-529a-4b72-9bf1-e8c27c3ff014 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.359547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.360653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.361247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.361613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.362447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:18.362874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.364451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.384002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 408/829] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:18.389356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.499947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18942f1-27be-4a43-b909-b8dc410e5f47 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.510838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.536910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.537546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.537826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.538095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.550379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.551871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.553600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:18.571634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:18.572608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:18.573710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:18.573710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:18.573710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:18.573710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 422/830] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:18.579857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.668884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820510-f6da-43fc-b0f9-0d1ad44c0388 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.708878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.709508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.709836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.710129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.717972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.720205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac778bd9-0e9a-472f-a32a-10e8b92164b9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.722429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:18.727580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:18.730632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:18.731498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:18.738667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 409/831] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:18.738667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.762186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.762839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.763117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.763449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.764757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:18.765069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.767414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.783642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 423/832] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:18.790583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.815677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:18.863704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91071b-4520-4ad4-8503-cb65f51f4d6f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.900417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.901470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.902195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.902868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.903702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:18.904142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.904497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d3cd62-dddf-49c5-8566-7f2acf210ffa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:18.906001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b18cb6-12e3-4ebf-a2d2-31a1e0e49d65 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.921271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 410/833] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:18.921271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b744f-7ef8-4334-8b1e-f52f0d764e40 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.921271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b744f-7ef8-4334-8b1e-f52f0d764e40 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.921271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b744f-7ef8-4334-8b1e-f52f0d764e40 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.922501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 411/834] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:18.923777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b744f-7ef8-4334-8b1e-f52f0d764e40 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:18.949085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:18.949965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.950240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.950515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.958267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:18.959742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ff5f22cf-3d57-42a0-93eb-f1f0bff44814 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:18.972036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:18.973126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:18.974197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:18.977831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 424/835] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:18.978925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:18.979286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:18.979563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:18.998143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:18.999351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:19.000068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.044779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b744f-7ef8-4334-8b1e-f52f0d764e40 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.075231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 425/836] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:19.075231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.082948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.091912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.092752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.095975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.095975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.107849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.109366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:19.124895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:19.126268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:19.127663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:19.127663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:19.127663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:19.131111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 412/837] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:19.132030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.187634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e9c2c4-69b4-4b4e-a98e-8ef821094369 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.225152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.226000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.226453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.226883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.227623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:19.227929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.229463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.249646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 426/838] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:19.261625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.267526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c061-47d4-4919-a591-b4e3b4436961 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.302602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.306886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.307749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.308191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.310151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.310151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:19.310151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.311428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.325223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 413/839] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:19.332585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.344468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.383372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43903e3-d87e-4dd0-9326-45e77ed286c5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.418305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.418965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.419286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.419565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/711f1b63cfa1461da6b5b9bb286c6f95/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.435861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=711f1b63cfa1461da6b5b9bb286c6f95, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.437339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.456245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19592122-29e9-428c-a621-71bd8f2648a6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.491995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.492620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.492896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.493167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.500352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.502383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.511528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 427/840] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:19 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/711f1b63cfa1461da6b5b9bb286c6f95/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:19.512788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:19.524279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:19.525310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:19.526504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 414/841] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:19.537613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.585832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.647527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5007e1f-69ae-4f6d-827f-19a263f95efb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.673074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.681943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c068b1-46fb-443d-acb6-ae7a1bebd7ef None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.692515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.693958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.694411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.694827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.695927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.697829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:19.708444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:19.711712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:19.712552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 428/842] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:19.715717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.730928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.731905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.732364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.732835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.733745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:19.734230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.736001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.764846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 415/843] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:19.771290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.840159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6a15-94eb-4190-a277-51b2ff9dcc44 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.878422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:19.881818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.882661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.883253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.883750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.890337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:19.890337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:19.890337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:19.890602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e8ea6-6cea-4465-85b4-40fc9512136d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:19.896618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 429/844] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:19.897817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:19.932805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:19.933436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:19.933715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:19.933985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:19.954297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.011341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.012721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aeda4a-b6b2-46ee-a8b3-5df41f0deb67 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.012839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f399d6-f540-4ecb-940b-6520b7f3a8e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.048924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.049537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.049812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.050080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.050709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:20.051004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.052425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.057885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 430/845] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:20.115021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.131444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 416/846] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:19 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:15:20.133138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.253840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc64092-cd2f-44a7-9663-10f96d7656c3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.274219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.296812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.298371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.298945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.299503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.308080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd629b-65e1-4a38-984e-e755b707a54b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.308358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd629b-65e1-4a38-984e-e755b707a54b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.308631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd629b-65e1-4a38-984e-e755b707a54b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.313018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.315543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.318118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 417/847] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:20.319821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.320096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.320362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:20.334029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:20.335034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:20.335962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:20.335962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:20.335962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:20.336704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 431/848] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:20.342384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.343663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:20.345009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:20.423646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 418/849] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:20.473569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d2528-7cbb-49d4-97fc-82fb7d92a518 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.508776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.509387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.509664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.509929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.510567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:20.510860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.512244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.525115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 432/850] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:20.530650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.638409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57494c-6c95-4c79-a591-db34f7b5f5b3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.660332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.675824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.676827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.677314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.677791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.686438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.689559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.693984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:20.704272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:20.705360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:20.706242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:20.708285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 419/851] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:20.714534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.728275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.812637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6da-9f6f-43c9-a42e-f2a0341c518d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.847049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a72606-2b2f-4411-b6f2-2ba3adffdce2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:20.850080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.850550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.850550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.850743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.851365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.852724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:20.862397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:20.863555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:20.864451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 433/852] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:20.869427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.887797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:20.888681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:20.889161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:20.889624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:20.890500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:20.890997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:20.893556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:20.906054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 420/853] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:20.912137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:20.972385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:20.988829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:21.008022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a67b97-f28b-4a43-b725-6e4646714942 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:21.042977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123b5c5-bee7-4728-ad23-a7bb24519acd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:21.057429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:21.058537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:21.058982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:21.059769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:21.060833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:21.061279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:21.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:21.069854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 434/854] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:21.075779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:21.083992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:21.084805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:21.085227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:21.085640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:21.086514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=d247a8be3add4804b86dcc0cd4cb72e3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:21.086986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:21.088678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2f2da-1158-4c05-ad1e-a34c6144d046 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:21.095698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 421/855] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:20 2022] GET /identity/v3/role_assignments?user.id=d247a8be3add4804b86dcc0cd4cb72e3&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:21.204168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5574dd-ad82-45c3-b88c-f4ef1e20ac37 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:21.249887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:21.250510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:21.250783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:21.251957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:21.252541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:21.252837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:21.254199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944356d1-484c-4bc7-afec-771167bfca0d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:21.263246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 435/856] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:21.566238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af49e-4288-4177-b903-5553ab7bd612 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:21.566629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af49e-4288-4177-b903-5553ab7bd612 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:21.566831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38af49e-4288-4177-b903-5553ab7bd612 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:21.568436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 422/857] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:21.574011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5833d4-ef51-446d-b0fd-a2bdc0295458 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:21.574302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5833d4-ef51-446d-b0fd-a2bdc0295458 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:21.574571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5833d4-ef51-446d-b0fd-a2bdc0295458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:21.596794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cd5833d4-ef51-446d-b0fd-a2bdc0295458 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:21.598409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cd5833d4-ef51-446d-b0fd-a2bdc0295458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:21.610305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:21.671941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 436/858] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:21.944447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af49e-4288-4177-b903-5553ab7bd612 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.024101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ee9-f727-4be8-9b36-077da57b40b2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.024648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ee9-f727-4be8-9b36-077da57b40b2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.024648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae68ee9-f727-4be8-9b36-077da57b40b2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.026229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 437/859] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:22.031914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.032202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.032673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.055199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:22.056676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:22.059376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38af49e-4288-4177-b903-5553ab7bd612 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.064111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:22.104116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.104748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.105023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.105292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.118128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:22.123858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:22.124769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:22.125634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 423/860] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:22.132549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.151250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 438/861] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:22.260821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec75fc6-0e59-4a93-9bba-29e72aeb00a9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.296888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.297570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.297877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.298779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.299814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:22.300114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:22.301597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:22.311569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 424/862] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:22.317173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.408069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:22.424666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.431440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f6df6d-4ed3-4504-a3b6-e7941e0bf2a1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.463562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:22.476976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.477621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.478040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.478432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.494228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:22.495976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:22.514780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:22.515831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:22.517490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:22.517490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:22.517490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:22.519256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 439/863] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:22.524073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.539613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e017c0-91fb-4e9b-b3b5-da3b457ded52 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.580007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.580861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.581244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.581615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.582472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:22.584019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:22.643016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f47b4-bc77-4562-b301-304ca831143b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.655231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 425/864] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:22 2022] POST /identity/v3/services => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:22.681961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:22.698964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.699788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.700065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.700331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.701036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:22.701336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:22.702702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:22.722456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 440/865] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:22.724519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:22.833324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:22.849321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8217d0f-25a4-4178-9e86-81197d4ce05f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:22.886227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:22.886868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:22.887209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:22.887448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:22.894876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:22.896376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:22.909502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:22.910620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:22.911798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:22.914204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 426/866] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:22.920399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.039474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db681f1-625d-4719-b9b9-89c5ffae2a2b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.076367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.076979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.077259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.077533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.078270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:23.078575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.080335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f3b7c-b95e-45f2-b7ef-0dea7ec90fc1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.099666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 441/867] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:23.105387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.201166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194826-4a32-4875-b6b2-579552be5f00 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.201612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194826-4a32-4875-b6b2-579552be5f00 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.201988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194826-4a32-4875-b6b2-579552be5f00 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.203711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 442/868] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:23.209640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5a9-417f-40c0-8207-5ddce417109b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.210020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5a9-417f-40c0-8207-5ddce417109b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.210390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd5a9-417f-40c0-8207-5ddce417109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.217979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40eab4-f06c-4835-9b5c-d3a7e70b1604 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.229821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-aa8bd5a9-417f-40c0-8207-5ddce417109b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:23.231184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aa8bd5a9-417f-40c0-8207-5ddce417109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:23.265277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.266058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.266440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.267405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.276608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.286328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.315888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 443/869] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:23.317230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77603b-f43f-48be-87da-7a75c3e60d2c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.317518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77603b-f43f-48be-87da-7a75c3e60d2c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.317788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77603b-f43f-48be-87da-7a75c3e60d2c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.322958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 444/870] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:23.324352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.324728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.325092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.344035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:23.345702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:23.350838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.352505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.423517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 445/871] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:23 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:23.477397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 427/872] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:23.478838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.584750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595ba28-7a89-4bec-97df-f8c1ab3557d3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.586625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.591295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.626617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.627235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.627475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.627750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.640983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.642586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.653387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:23.663578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:23.664648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:23.665970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:23.665970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:23.665970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:23.665970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 446/873] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:23.670922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.717599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6144a-3464-42a4-a2ed-f04d496ccf21 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.753059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.753700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.753978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.754250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.754833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.756259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:23.765396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:23.766383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:23.767486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 428/874] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:23.768337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.792355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ec34b-1a5e-4bf3-90e5-2ae62d8be7c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.830941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.831576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.831945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.832120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.832768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:23.833066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.834427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.846678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 447/875] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:23.847529 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:23.864644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.878345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53751cdf-ac92-4568-9b59-222bc638f0b9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.889367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:23.914905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:23.915554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:23.916101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:23.916630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:23.917750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:23.919638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:23.964772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246d939-2ff4-492e-a7ec-8bee74070b9c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:23.995274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 429/876] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:23 2022] POST /identity/v3/services => generated 269 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:23.995274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:24.020923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:24.021604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.021895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.022168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.022901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:24.023231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:24.025562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:24.034344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 448/877] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:24.039785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:24.119478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d3d72-f6c5-48b5-83c9-8ad2945b573d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:24.150983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:24.174092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:24.180179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.180179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.180179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.180403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef60a89-dd7f-4ea1-8dc6-feba990d1896 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:24.184203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:24.184902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:24.202290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:24.203454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:24.205277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 430/878] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:24.211724 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:24.223793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:24.225199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.225199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.225199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.225663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:24.225967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:24.227416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:24.232537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 449/879] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:24.259886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:24.342945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ec398-f868-4300-bd80-402352578a14 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:24.382542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:24.383194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.383465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.383741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.384450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:24.384757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:24.386203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b2a7f-abc2-4db7-83cf-0660fa3c6007 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:24.398734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 431/880] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:24.404798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:24.515251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd919b-9eec-408c-ada3-3a5e3a47fc95 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:24.556829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:24.557801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.558310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.558783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.559782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=711f1b63cfa1461da6b5b9bb286c6f95) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:24.560306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:24.561942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ab8b5-757b-4bd2-83a6-2a32b71a28ca None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:24.569287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 450/881] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:24 2022] GET /identity/v3/role_assignments?user.id=711f1b63cfa1461da6b5b9bb286c6f95&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:24.781354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bad10-4e76-47fb-99bc-855756990c73 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.781722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bad10-4e76-47fb-99bc-855756990c73 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.781923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488bad10-4e76-47fb-99bc-855756990c73 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.784136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 432/882] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:24.789615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:24.789900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:24.790176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:24.799480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:24.812605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:24.816505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:24.891266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 451/883] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:25.157641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bad10-4e76-47fb-99bc-855756990c73 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:25.268716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:25.279585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488bad10-4e76-47fb-99bc-855756990c73 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:25.316901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:25.317540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:25.317813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:25.318077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:25.318653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:25.320042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:25.329750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:25.330857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:25.331736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:25.335514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 433/884] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:25.337398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:25.444283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b326d7e-4397-4ba6-8964-56c3c6bb5ab3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:25.473420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:25.478636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:25.479455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:25.479932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:25.480566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:25.481367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:25.481826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:25.483392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:25.493461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 452/885] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:25.495488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:25.599278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680c514-a5dd-4154-8a1c-c6f63baff3de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:25.635423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:25.636334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:25.636869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:25.637315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:25.638208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:25.638669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:25.640236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:25.650830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 434/886] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:25.652409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:25.763490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1df05-68c3-4581-b8ed-34fc61572823 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:25.802827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:25.803714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:25.803991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:25.804256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:25.804958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] RBAC: Adding query filter params (interface=public, service_id=c817a9923a8743d2b2e50e697722ffdc, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:25.805260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:25.806612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23620384-1a43-40c8-8dc7-3f13fe5f4f7e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:25.812189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 453/887] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:25 2022] GET /identity/v3/endpoints?service_id=c817a9923a8743d2b2e50e697722ffdc&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:25.818066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:25.838516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:25.925607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a26472-9ed3-4bd0-abe7-ce17958e8baf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:25.962426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:25.963435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:25.963911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:25.964396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:25.965187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:25.966786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d00be-d3fb-46c7-a2d8-9847b644c10f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:25.978438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 435/888] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:25 2022] GET /identity/v3/services => generated 1268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:15:26.193215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0666-51d1-4047-9c83-3be5c23c3a9f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.193611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0666-51d1-4047-9c83-3be5c23c3a9f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.193785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa0666-51d1-4047-9c83-3be5c23c3a9f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.195389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 454/889] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:26.200782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.201281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.201362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.220860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:26.222062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:26.248072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6229595-01fd-4729-afb2-f935925a7cb6 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.248652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6229595-01fd-4729-afb2-f935925a7cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.249271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6229595-01fd-4729-afb2-f935925a7cb6 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.250912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 455/890] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:26.256638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ccecd-22c6-4992-a3ef-6d93936d1811 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.256981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ccecd-22c6-4992-a3ef-6d93936d1811 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.257326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ccecd-22c6-4992-a3ef-6d93936d1811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.277837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-386ccecd-22c6-4992-a3ef-6d93936d1811 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:26.279172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-386ccecd-22c6-4992-a3ef-6d93936d1811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:26.304911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 436/891] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:26.349245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:26.362354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 456/892] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:26.592604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:26.608566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:26.646913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87bf3-d207-417c-822f-6646e6985140 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.647302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87bf3-d207-417c-822f-6646e6985140 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.647575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a87bf3-d207-417c-822f-6646e6985140 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.648894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 457/893] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:26.649627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a87bf3-d207-417c-822f-6646e6985140 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:26.702192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877298d-584c-48a1-a1b3-248ecfa3af20 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:26.737094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:26.737722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.737999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.738265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.738849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:26.740263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a5be171e-3d70-4e41-904e-a811f9573ed1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:26.749093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:26.750035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:26.752794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 437/894] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 16:15:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:26.753586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.753586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.753586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.764793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a87bf3-d207-417c-822f-6646e6985140 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:26.771187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:26.772401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:26.802997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:26.803885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:26.804178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:26.804438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:26.805090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:26.806441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:26.840459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 438/895] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:26.841363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:26.851775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:26.885028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 458/896] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:26 2022] POST /identity/v3/services => generated 241 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:26.966262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baee754-d78e-4891-b667-17a8a12f6ee6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.010978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.011628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.011903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.012170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.012858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:27.013160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.014573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.020841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 439/897] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:27.026369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.086824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.127356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.143891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6601a5-61d0-4061-b533-51ca156fc614 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.181979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.183034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.183468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.183906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.184730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:27.185182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.186808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.197979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 459/898] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:27.199397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.199866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.238334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686717f7-a75f-4bbb-aa1f-494835e49d70 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.273502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.274137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.274411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.274679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.286306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:27.291856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:27.292998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:27.294228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:27.298017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 440/899] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:27.299755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.326436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e055740-c8be-4eaa-8f05-5056a797ed3f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.345498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.367281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.367930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.368220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.368497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.369533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] RBAC: Adding query filter params (interface=public, service_id=65565df61744470484c2a75eb52e7716, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:27.370067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.372566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.380096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 460/900] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/endpoints?service_id=65565df61744470484c2a75eb52e7716&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:27.385380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.425256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9430d45-b204-4c16-8b17-2fc690c75546 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.455511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.467516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.468475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.468752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.469025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.470505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:27.471531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.473467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.487991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 441/901] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:27.494561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.521775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9a320-f84d-4bd0-955b-1638c2913022 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.559734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.560363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.560703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.560970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.561718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.563124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.569805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 461/902] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/services => generated 1497 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:15:27.619769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999b725-aa42-4b2d-8b73-0321395b0ddf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.660388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.661060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.661326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.661594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.675084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.676609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:27.695664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:27.696691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:27.697599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:27.697599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:27.697599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:27.701771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 442/903] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:27.703480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.758459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.822933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c006-49c8-4e6e-af85-e855baa3b426 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:27.859793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:27.860395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.860699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.861152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.861989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:27.862291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:27.863724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2df45-bc7e-46f2-8b8b-cdde5fe6919e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:27.879276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 462/904] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:27.882279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:27.931287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:27.940637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871254-0b5f-4ba1-93ff-cedb1916984a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.940918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871254-0b5f-4ba1-93ff-cedb1916984a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.941189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4871254-0b5f-4ba1-93ff-cedb1916984a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.950601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 463/905] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:27.950601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:27.950601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:27.950601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:27.970886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:27.972923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:28.006393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba094ede-2e3d-4ca7-8db1-85371bc499f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.047381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 464/906] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:28.053994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.054687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.054966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.055257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.063589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.065529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:28.077327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:28.078279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:28.079284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:28.083270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 443/907] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:28.084912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.117805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.197639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa81a7-00c0-4be7-a458-43d36de029ea None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.233116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.233946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.234400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.234814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.235777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:28.236229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.237789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.257002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 465/908] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:28.265107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.372762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.374245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8353dd-0254-4418-a03d-ea0ac347b2e5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.382350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.429222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.430223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.431172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.431650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.495503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.509102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184485f-cd6f-48b8-82be-ad2668dfbaaf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.516352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.518778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.559597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.560554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.561527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.561975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.562732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.565004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:28.574257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:28.575282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:28.580706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 466/909] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:28.583362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.635703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.639916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 444/910] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:28.656299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.724121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd143cf5-35e7-43ad-965d-54d8227d817d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.766894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07389848-d2bd-4f0b-9c1f-e4e326441f89 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.771898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.772978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.773459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.773924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.774768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:28.775296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.777469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.790565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 467/911] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:28.792098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.807964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6093ab-628f-450c-b294-284169909851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.808594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6093ab-628f-450c-b294-284169909851 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.808868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6093ab-628f-450c-b294-284169909851 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.809135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6093ab-628f-450c-b294-284169909851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.819629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.820625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6093ab-628f-450c-b294-284169909851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.822304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6093ab-628f-450c-b294-284169909851 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ea6093ab-628f-450c-b294-284169909851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:28.838389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:28.839687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:28.840775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:28.840775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:28.840775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:28.847144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 445/912] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:28.848389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6093ab-628f-450c-b294-284169909851 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:28.909258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:28.924131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbe7a-b757-4193-b8b7-19fce60e9ef7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.963317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6093ab-628f-450c-b294-284169909851 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:28.968727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:28.969507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.970452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.971033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.972488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:28.972940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:28.974884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d506c7-ccbe-4147-af0d-5d5ca489efe1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:28.980545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 468/913] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:28.981957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86490-de84-4c66-a8a7-d3bbf7a9e617 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:28.982244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86490-de84-4c66-a8a7-d3bbf7a9e617 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:28.982495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd86490-de84-4c66-a8a7-d3bbf7a9e617 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:28.983703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 469/914] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:28.986098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd86490-de84-4c66-a8a7-d3bbf7a9e617 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.007410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.008397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.008999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.009339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.009982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:29.010294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.011795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1217682-85ef-45f7-a7c6-72475ceac841 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.030432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 446/915] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:29.030432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.030432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.030432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.048776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:29.050086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:29.083705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:29.102905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd86490-de84-4c66-a8a7-d3bbf7a9e617 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.122574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 447/916] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:29.123746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.145865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.146524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.146803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.147069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.148025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.149759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.178162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:29.237982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69174c93-b773-42eb-9564-3c1bc3d7e478 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.248690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 470/917] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:29.289409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.290060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.290303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.290576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.299624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.301207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:29.314606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:29.315677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:29.317300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 448/918] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:29.321816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.433012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.444331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445384b2-9b39-4a6e-92d4-dbc49ce425b1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.488159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.488789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.489137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.489329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.490328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:29.490328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.491804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.499428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:29.505533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 471/919] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:29.513303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.562396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:29.568476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671651d3-b417-4a86-98d8-2856fc820137 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.614104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.615273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.616062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.616821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.617605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.619668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.621667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5b4ad-c655-41fd-b098-80b168f27c85 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e8542738-3d47-4cba-b47c-03ac2296cf87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:29.630164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:29.631245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:29.632137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 449/920] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:29.632920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77b5d-08e7-4bae-8b8b-2c394cb3bd71 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.633977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77b5d-08e7-4bae-8b8b-2c394cb3bd71 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.633977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77b5d-08e7-4bae-8b8b-2c394cb3bd71 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.634749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 450/921] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:29.639742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a77b5d-08e7-4bae-8b8b-2c394cb3bd71 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.665962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.666571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.666848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.667117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.667878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] RBAC: Adding query filter params (role.id=4726495c16694f73a5adbe329cb1586c, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=711f1b63cfa1461da6b5b9bb286c6f95) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:29.668185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.669606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281671e4-5865-4e99-b029-b6dd16256cbc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.683195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 472/922] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/role_assignments?user.id=711f1b63cfa1461da6b5b9bb286c6f95&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=4726495c16694f73a5adbe329cb1586c => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:29.683195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.683195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.683195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.698011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:29.699290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:29.771111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:29.779092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a77b5d-08e7-4bae-8b8b-2c394cb3bd71 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.795558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 473/923] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:29.825364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.825917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.826243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.826521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.827249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:29.827530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.829063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.835283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 451/924] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:29.840806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:29.952742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc506efc-7666-424d-a90a-304ff1132dae None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:29.989250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:29.989925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:29.990209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:29.990480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:29.991116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:29.991442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:29.992819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:29.998194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 474/925] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:30.004432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.037108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:30.070330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.112991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da16e34-c935-4b3c-9066-a93dd33c4eea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.132219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:30.151064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.152019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.152495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.152958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.154270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] RBAC: Adding query filter params (interface=public, service_id=00f723fdca0b46fd9127e532d475a908, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:30.154775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.157065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.165321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 452/926] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:30 2022] GET /identity/v3/endpoints?service_id=00f723fdca0b46fd9127e532d475a908&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:30.172287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.185186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598940bf-31d7-41ef-a6b3-2aa601c7beca None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.233070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.233746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.234023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.234288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.234864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.236429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:30.245499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:30.246478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 01 16:15:30.247434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:30.252034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 475/927] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Sep 1 16:15:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:30.253274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.291479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:30.305748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7228a8c-1eef-4172-b406-991d4e045e69 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.350912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.352042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.352566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.353342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.354144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.356152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.364987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 453/928] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:30 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:15:30.371411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d3eb2-3644-43a3-9de9-a592b71fdd3e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.395180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:30.411751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.412765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.413192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.413643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.414484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:30.414955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.416702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.423505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 476/929] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:30.429312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.545550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9abba8f-1a37-4a1d-968f-a1c6cabc5922 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.580688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.581299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.581634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.581846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.582464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:30.582761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.584153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96a32d-28f8-4902-ab7c-74aa98c9f462 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.589981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 454/930] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:30.595954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:30.699675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8edf7-b76c-43ba-bde8-5c5209ea3e3c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:30.734917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:30.735572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:30.735889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:30.736149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:30.736834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:30.738194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22439393-c77d-4006-a91d-f6b97334a2eb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:30.833544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 477/931] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:31.281360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae4059-b931-4279-84d5-edf1e8fe2e50 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:31.281708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae4059-b931-4279-84d5-edf1e8fe2e50 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:31.281900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae4059-b931-4279-84d5-edf1e8fe2e50 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:31.284038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 455/932] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:31.289160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df9505-c414-449b-b860-28511e07969a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:31.289439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df9505-c414-449b-b860-28511e07969a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:31.289710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df9505-c414-449b-b860-28511e07969a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:31.308595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f3df9505-c414-449b-b860-28511e07969a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:31.309767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3df9505-c414-449b-b860-28511e07969a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:31.347969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:31.391261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 478/933] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:31.657784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae4059-b931-4279-84d5-edf1e8fe2e50 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:31.679116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:31.744269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f11b-540d-46dc-be7c-380e7018dd26 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:31.744829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f11b-540d-46dc-be7c-380e7018dd26 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:31.744829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5f11b-540d-46dc-be7c-380e7018dd26 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:31.746373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 479/934] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:31.751263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:31.752095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:31.752553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:31.772049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:31.773368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:31.777659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae4059-b931-4279-84d5-edf1e8fe2e50 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:31.812601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:31.813456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:31.814431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:31.814861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:31.815767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:31.817372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:31.820011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING keystone.server.flask.common [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 01 16:15:31.846464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 480/935] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:31.865628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:15:31.865628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:15:31.865628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [parameters: {'id': '63ebd28f78d640dbb6e524c0c456b7a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:15:31.865628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 63ebd28f78d640dbb6e524c0c456b7a8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 63ebd28f78d640dbb6e524c0c456b7a8. Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:15:31.875214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 01 16:15:31.876886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Sep 01 16:15:31.877908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:15:31.878891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:15:31.885610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:15:31.886717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': '63ebd28f78d640dbb6e524c0c456b7a8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:31.887685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:31.889558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 63ebd28f78d640dbb6e524c0c456b7a8. Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:31.890566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 456/936] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Sep 1 16:15:31 2022] POST /identity/v3/projects => generated 262 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:31.891178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:31.992837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:31.999358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05300d70-637c-4c56-a634-3dae83b7bbb3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.037038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.038120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.038889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.039348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.046468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:32.048111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:32.060969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:32.062073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:32.063259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:32.065371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 481/937] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:32.071715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.145058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.152401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:32.188644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b22d4-d98b-4910-a88c-467838a369c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.228220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.228832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.229105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.229373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.230042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:32.230339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:32.232024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:32.259796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 457/938] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:32.272919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44235f-2259-4a0d-9999-25418a22a521 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.314894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.315945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.316426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.316924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.330282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:32.336641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:32.337635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:32.338508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:32.341436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 482/939] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:32.348638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.462362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49bd30-632a-4c4c-8d88-14c30486cb1d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.500695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.501244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.502111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.502474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.505693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:32.505693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:32.505693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3cd07-bdca-4212-949b-b122ce2db838 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:32.506691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:32.514764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 458/940] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:32.521204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.521705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2a47-6e2b-401b-95f5-8472d0456a05 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.522066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2a47-6e2b-401b-95f5-8472d0456a05 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.522262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2a47-6e2b-401b-95f5-8472d0456a05 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.524000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 459/941] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:32.530215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.535731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.535731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.551540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:32.552667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:32.554922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:32.641479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8e8d5-8b77-4e6f-af00-79b6425cbd98 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.648811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 460/942] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:32.683328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.683962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.684238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.684509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.695842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:32.697292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-10394b13-9f93-4243-80a8-55a7fb6f17ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:32.712628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:32.713631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:32.714521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:32.714521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:32.714521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:32.714521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 483/943] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:32.719725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.833800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effd5b9-060c-42ed-a9b5-5fc6ebef09e3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:32.854999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b341a-410c-466a-8837-13325fedc4b0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.856001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b341a-410c-466a-8837-13325fedc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.856481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b341a-410c-466a-8837-13325fedc4b0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.858251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 484/944] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:32.861811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:32.864139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.864660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.865125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.870817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:32.871911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:32.872122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:32.872334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:32.872979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:32.873279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:32.875639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:32.885678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:32.887193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:32.893269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 461/945] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:32.895683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:32.960006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:32.971856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 485/946] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:32.971856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.013969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e70506-18a4-4d2e-ac1c-a29af0cc7178 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.049570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.050200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.050481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.050751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.058874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.060365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:33.072370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:33.073584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:33.074574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 462/947] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:33.079984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.084105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd061a9-9d54-460d-b390-efd73fcc1d39 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.122387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.123380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.123881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.124376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.125227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.127061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.137449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:33.140948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:33.147295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:33.148205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 486/948] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:33.153206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.202880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ddb68-0dec-46f9-8641-0831242e229f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.224840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.239114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.239756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.245252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.245537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.246523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.246634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.248102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.271473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3ef8-57b6-4ead-9fc1-68bf1f801f6b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.274883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 463/949] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:33.276055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.316695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.317737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.318014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.318282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.319374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.319679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.321718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.328454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 487/950] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:33.329270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.378703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307e431-f102-43d9-98df-9f9243342908 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.381784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.415506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.416515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.416965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.417389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.418122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.419679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:33.430040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:33.431207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:33.432069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:33.432813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 464/951] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:33.433276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.443864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ac6e8-9a31-49c0-a522-b40d883a7005 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.473115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.481922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.482555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.482831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.483097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.493627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.495070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:33.515681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:33.516691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:33.518310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:33.518310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:33.518310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:33.523630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 488/952] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:33.524253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.548035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8d9e3-bcd0-45ba-9744-125e064cf25b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.585018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.585655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.585940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.586217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.586844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.587170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.588542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.593761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 465/953] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:33.594540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.629014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.637306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c1aeb-362a-4bcf-bc52-ee46525b87eb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.673592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.674303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.674654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.674998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.675779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.676223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.677741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.683019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 489/954] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:33.683818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.707082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac9606-4198-4a39-bfab-7c4d7d25f1db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.729956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.743997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.744744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.745093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.745516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.746253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.746622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.748097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.764405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 466/955] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:33.764405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.792304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab70f4-1df6-49f2-8ece-24cb41623592 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.831622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.832822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.833199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.833543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.834487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.836922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.858740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:33.867380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d4393-2ac4-4007-acf0-4cef74d2262a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:33.900894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:33.901426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:33.901706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:33.901992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:33.902600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:33.902899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:33.904343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:33.909573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 467/956] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:33.910418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:33.935850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 490/957] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:34.048277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26b3f5b-a2f6-414c-b99b-71eec25f0405 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:34.101842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:34.102486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.102764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.103036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.110327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:34.111968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bd4fd99b-d712-4c5a-9e69-572bffa703a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:34.123792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:34.124709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.125595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 468/958] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:34.133459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:34.241229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:34.244462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40810910-3c52-4a51-ba7c-3dc7970fc667 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:34.280871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:34.281486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.281765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.282034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.282763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:34.283077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:34.284750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:34.296689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4e79-d25a-42d4-bf54-aa69c6329379 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.296974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4e79-d25a-42d4-bf54-aa69c6329379 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.297238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d4e79-d25a-42d4-bf54-aa69c6329379 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.299289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 469/959] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:34.301360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 491/960] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:34.306454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.306738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.307002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.308127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:34.328470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:34.330318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:34.338607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:34.416383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 470/961] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:34.432694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68599ea6-1067-4311-9afb-c7acf9f10270 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:34.469885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:34.471333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.471826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.472291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/711f1b63cfa1461da6b5b9bb286c6f95/roles/4726495c16694f73a5adbe329cb1586c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.474706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=711f1b63cfa1461da6b5b9bb286c6f95, role_id=4726495c16694f73a5adbe329cb1586c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:34.476383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:34.559995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 492/962] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:34 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/711f1b63cfa1461da6b5b9bb286c6f95/roles/4726495c16694f73a5adbe329cb1586c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:34.690498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:34.797265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6db598-c755-45aa-acb3-5c47089b1c10 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:34.832680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:34.833211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:34.833533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:34.833811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:34.834471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:34.835929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:34.878853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:15:34.878853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:15:34.878853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [parameters: {'user_id': 'c160d4351bed4f1ebd7c61de5af31709', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:15:34.878853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:15:34.886166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:15:34.887021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 01 16:15:34.888051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 01 16:15:34.888948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 01 16:15:34.889849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 01 16:15:34.890738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c160d4351bed4f1ebd7c61de5af31709', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:34.893199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 01 16:15:34.895037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:34.901074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 471/963] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Sep 1 16:15:34 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:34.904241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:34.924931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:35.014349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a031b5-c7da-4873-83d3-4cc77560760e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:35.051114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:35.052145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.052642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.053115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:35.060838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:35.062517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b6fe8f68-6928-4cda-85bd-1d2b5c5a53cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:35.074818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:35.075834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 01 16:15:35.076864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:35.079420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 493/964] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:35.087270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:35.091617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:35.200117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f54758-604e-4e1f-899f-b3714ae6e22d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:35.235514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:35.236670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.237056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.237333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:35.238030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:35.238332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:35.239986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15559ef1-1411-4548-b0ad-cf8acdef934b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:35.281568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 472/965] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Sep 1 16:15:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:35.960614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010dd1-1488-40fb-a88f-087b2911c7a3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.960984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010dd1-1488-40fb-a88f-087b2911c7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.961694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93010dd1-1488-40fb-a88f-087b2911c7a3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:35.963067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 494/966] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:35.968213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.968694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.968774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:35.988202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:35.989765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:35.991938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b77a7-835c-47c8-88fa-280636198ff5 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.992221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b77a7-835c-47c8-88fa-280636198ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.992487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b77a7-835c-47c8-88fa-280636198ff5 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:35.993966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 495/967] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:35.998806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:35.999081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:35.999378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.018021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:36.019223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:36.053763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:36.064750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 473/968] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:36.094778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 496/969] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:36.332637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.362016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.374270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:36.442236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceee5f8-8189-4d51-9e8b-583f9c59439b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:36.465412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05f6a-b9d7-46f1-bcc1-400a7b6067d8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:36.480399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:36.481012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.481289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.481560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.482224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:36.483633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca30ec2-62ec-432a-b811-6a633459c9f1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:36.502124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:36.502124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.502124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.502124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.502817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:36.504221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:36.512919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:36.513915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:36.514755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 497/970] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:36.520816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.541561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 474/971] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:36 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:36.597064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58e18-2322-437b-ad4d-263db44089ca None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.597387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58e18-2322-437b-ad4d-263db44089ca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.597617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b58e18-2322-437b-ad4d-263db44089ca None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.599122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 475/972] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:36.603551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:36.606860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.607144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.607466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.631570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:36.632943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:36.636757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df45c17-891b-4d4a-a1a3-1fb2cc3a149a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:36.686232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:36.687114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.687575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.687844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.688481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:36.688777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:36.690176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:36.692501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:36.695797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 498/973] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:36.701805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.718704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 476/974] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:36 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:36.808270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ed8b1-a968-463b-b2ab-d95968fa52c0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:36.837146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:36.842782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:36.843421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.843700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.843993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.844596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:36.844889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:36.846281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:36.851283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 499/975] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:36.856945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.958129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef003fb2-3823-4838-97f6-3bb2ea61a3f3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:36.989623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:36.994836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:36.995484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:36.995760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:36.996049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:36.996770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] RBAC: Adding query filter params (interface=admin, service_id=00f723fdca0b46fd9127e532d475a908, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:36.997075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:36.998445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.005800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 477/976] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Sep 1 16:15:36 2022] GET /identity/v3/endpoints?service_id=00f723fdca0b46fd9127e532d475a908&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:37.010915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.061717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:37.103138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513cdd3e-4c43-45c9-a04d-8e3377d49c9b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.124585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d0161-18e5-4cc5-8f53-d0b8ab57032e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.141847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.142951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.143437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.143963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.155699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:37.159021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.159667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.159944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.160214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.160816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.162176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:37.163091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:37.164205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:37.165239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:37.167212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 500/977] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:37.171263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 478/978] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:15:37.173409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.294097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784c4a-2fc3-438b-a4de-08cd215422c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.307080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:37.332702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.333235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.333526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.333773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.334431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:37.334729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.336121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.346215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 479/979] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:37.351996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.410482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:37.457177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b284387-5085-4e02-9939-888e60e2029a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.491986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.492622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.492900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.493171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.504605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.506043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:37.520414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:37.521986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:37.523652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:37.523652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:37.523652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:37.523652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 501/980] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:37.527407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.631272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc968052-7812-4400-947c-b88d1aaf4319 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.666975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.667630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.667907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.668174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.668832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:37.669158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.670565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.683358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 480/981] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:37.688878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.790343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902ab1e-76b7-4580-92ac-3d259fa6d066 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.799697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:37.824455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.825074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.825347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.825614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.833236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.834672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:37.847190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:37.848384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 01 16:15:37.849541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:37.851923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 502/982] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:37.857797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:37.891453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:37.960862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56bba0-383a-43c4-a171-af5094eeed05 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:37.996187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:37.996961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:37.997096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:37.997371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:37.998095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:37.998400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:37.999869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:38.022029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 481/983] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Sep 1 16:15:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:38.023367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.127604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20799bf3-b775-4c78-a097-68d6191811c2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:38.162654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:38.163661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.164164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.164676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.206314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:38.250489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:38.252607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a92ef2f-5591-4dec-8b06-3d67fd5f4d9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:38.356045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 503/984] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:38.362090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.456553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:38.466294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b77226-e215-4b67-bdeb-4a8f3b0dbe8e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:38.502297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:38.502952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.503262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.503537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.513669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:38.515122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:38.519373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd8c8e-d9fa-484d-8ac4-b57990bd5e96 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.519658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd8c8e-d9fa-484d-8ac4-b57990bd5e96 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.519930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd8c8e-d9fa-484d-8ac4-b57990bd5e96 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.521399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 504/985] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:38.526285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.526565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.526835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:38.530539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:38.531514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:38.532441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:38.532441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:38.532441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:38.532441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 482/986] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:38.537918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.547456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:38.549454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:38.612156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:38.618212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 505/987] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:38.644587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656de08-11f7-4ac5-b355-3f88bed948ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:38.680980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:38.681586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.681860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.682127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.682801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:38.683096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:38.684482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:38.697425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 483/988] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:38.702788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.804746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f969a-766a-4ef3-a96b-a269f3b86231 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:38.839577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:38.840201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:38.840474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:38.840739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:38.847632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:38.849050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:38.859972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:38.860862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:38.862282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:38.867190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 506/989] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:38.867702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.870732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:38.889719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:38.952840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:38.979311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9762631-7d51-4718-92f9-121a2ab05ba6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:38.998115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb64f889-cbd2-4347-a0a2-f00fc98640f5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.017458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.018269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.018479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.018713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.019439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.019746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.021186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.033573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 484/990] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:39.036844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.037490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.037771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.038047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.038696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.039468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.040250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:39.050342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:39.051483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:39.052335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 507/991] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:39.057697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.108649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.150923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d137a3-cfb3-4c5b-a08a-6b1f1ff18216 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.164294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ce25-d50d-43ce-8346-a6e9188c5047 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.185558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.186167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.186455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.186731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.187501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] RBAC: Adding query filter params (role.id=4726495c16694f73a5adbe329cb1586c, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=711f1b63cfa1461da6b5b9bb286c6f95) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.187811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.189079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.189274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01dc8b-ab65-44bd-b2c7-c27b1c2cfdd9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.195562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 485/992] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/role_assignments?user.id=711f1b63cfa1461da6b5b9bb286c6f95&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=4726495c16694f73a5adbe329cb1586c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:39.199740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb35b-3ce3-49a7-b174-8fe08a46211d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.199740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb35b-3ce3-49a7-b174-8fe08a46211d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.199740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb35b-3ce3-49a7-b174-8fe08a46211d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.199740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 486/993] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:39.204250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.204529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.204844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.213994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.214895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.215749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.216209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.217230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.218096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.219543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.226103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 508/994] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:39.227404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:39.228660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:39.233168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.319456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 487/995] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:39.352536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20142864-c110-4c94-b7d1-6af48e14d70c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.394400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.395008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.395330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.395599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.396290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.396594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.398011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.403192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 509/996] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:39.409519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.468711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.511749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ac16a-4b73-4d03-82d2-5a4191387151 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.546600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.547249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.547529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.547798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.549045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] RBAC: Adding query filter params (interface=public, service_id=68fc2139360b455dba559cf6ef23938f, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.549431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.550810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.560659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 488/997] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/endpoints?service_id=68fc2139360b455dba559cf6ef23938f&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:39.561928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.599658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.618133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.676699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f45b-3da9-43be-a872-93a9135b34e1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.700706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c0919b-b80e-49bf-ac92-09eae1b1b1c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.720274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.721471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.721941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.722162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.722290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.722808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.724195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.731427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 510/998] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:15:39.748886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.749860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.750316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.750791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.751661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.753790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:39.764330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:39.765393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 01 16:15:39.766236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:39.771315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 489/999] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:39.774047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:39.885694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff81441-f78a-4ca9-9f6c-741c8aa1a3d5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:39.901703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:39.921996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:39.922662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:39.922937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:39.923269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:39.924664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:39.925002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:39.926367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:39.932151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 511/1000] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:39.937682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:40.038293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f390b-3ba8-4640-9481-39d23eebf1a2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:40.073212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:40.073818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:40.074094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:40.074361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:40.074989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:40.075318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:40.076681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23541f67-1af2-4fd6-b09d-30a6f26f5ce3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:40.086093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 490/1001] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Sep 1 16:15:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:40.087592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:40.189471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8074370-1711-417e-841d-1b46c39f15f3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:40.225935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:40.226540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:40.226815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:40.227083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:40.227788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:40.229307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe18f9-5a26-4991-9eb9-340a46383a60 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:40.324541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 512/1002] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:15:41.162618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2121-5f23-4c68-9648-99b73ffd42c5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.163170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2121-5f23-4c68-9648-99b73ffd42c5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.163170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc2121-5f23-4c68-9648-99b73ffd42c5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.164786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 491/1003] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:41.169884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e9a6-910c-44a3-8342-50f5f5fcc55e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.170171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e9a6-910c-44a3-8342-50f5f5fcc55e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.170445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54e9a6-910c-44a3-8342-50f5f5fcc55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.189983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4c54e9a6-910c-44a3-8342-50f5f5fcc55e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:41.191202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4c54e9a6-910c-44a3-8342-50f5f5fcc55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:41.221840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:41.272096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 513/1004] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:41.541602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc2121-5f23-4c68-9648-99b73ffd42c5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:41.593679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:41.646885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc2121-5f23-4c68-9648-99b73ffd42c5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:41.680353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef911e-3a90-46e9-811b-946192ceaee8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.680877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef911e-3a90-46e9-811b-946192ceaee8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.681325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef911e-3a90-46e9-811b-946192ceaee8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.682978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 514/1005] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:41.683596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:41.684245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.684522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.684789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.685370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:41.686748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:41.690742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.691234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.691685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:41.695869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:41.697028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:41.698114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 492/1006] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 16:15:41 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:41.702578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:41.715762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:41.716955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:41.796268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 515/1007] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:41.818942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54dbd2-c84f-4ebd-84e2-af4977c7eafb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:41.833107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:41.854787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:41.855709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:41.856195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:41.856647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:41.857499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:41.857977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:41.859600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:41.865838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 493/1008] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:41.871246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:41.968632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:41.979473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ea184-79f8-4b03-8389-e4b2f0c1c90c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.022547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.023171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.023461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.023733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.024564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] RBAC: Adding query filter params (type=image) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:42.025025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.027243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.039198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 516/1009] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:42.062207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.173990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fea9-c841-495f-a172-1f55c194f321 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.211027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.211708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.211990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.212261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.212847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.214221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cd815045-08c9-4ffa-8130-77ab02cd724e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:42.223275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:42.224284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:42.225829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 494/1010] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:42.230512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.276734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14d34-6ab5-467b-97c1-f0d76a6b757c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.277049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14d34-6ab5-467b-97c1-f0d76a6b757c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.277281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff14d34-6ab5-467b-97c1-f0d76a6b757c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.278752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 495/1011] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:42.284360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.284647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.284915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.303648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:42.305099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:42.347607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:42.349538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0929f-cee3-435b-90af-3130cbc1762d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.385350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 496/1012] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:42.396545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.397223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.397587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.397929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.398626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:42.398992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.400435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.410557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 517/1013] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:42.411815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.515441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9c5e3-d437-4b90-9613-2615da8b6fa7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.550175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.550793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.551065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.551380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.552017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:42.552319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.553697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.558979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 497/1014] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Sep 1 16:15:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:42.564607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.591007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:42.659249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.678144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f0b94-54a7-4e46-968d-c213ab1a215d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.701682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:42.718693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.719334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.719615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.719886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.720608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.722146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da06e064-2971-4cce-bae2-a2526024a672 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.773438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ccd24-356b-4f6f-b00f-335290447458 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.807813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.808439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.808716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.808984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.810066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.811528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:42.813094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 498/1015] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:42.819276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:42.832823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 518/1016] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:15:42.929236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b951f7f-0157-42d4-a2c6-972b407c4998 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:42.931108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:42.974449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:42.975297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:42.975409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:42.975602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:42.976702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:42.978406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7a7c6-0a2e-420d-8d86-052418d8b4ef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:43.057359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 499/1017] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:42 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:15:43.948551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f6b-b2f8-4673-a090-6f68c1331b61 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:43.949079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f6b-b2f8-4673-a090-6f68c1331b61 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:43.949079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7f6b-b2f8-4673-a090-6f68c1331b61 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:43.950570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 519/1018] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:43.955585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca233f03-9473-4acd-b973-aeff0be450f1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:43.955976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca233f03-9473-4acd-b973-aeff0be450f1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:43.956138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca233f03-9473-4acd-b973-aeff0be450f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:43.975042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ca233f03-9473-4acd-b973-aeff0be450f1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:43.976240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca233f03-9473-4acd-b973-aeff0be450f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:44.046374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 500/1019] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:43 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:44.318080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7f6b-b2f8-4673-a090-6f68c1331b61 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:44.397347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:44.419325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7f6b-b2f8-4673-a090-6f68c1331b61 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:44.455057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:44.455690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:44.455962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:44.456229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:44.456887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:44.458240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1cc83-42a8-47e0-80e2-e4c72f87e245 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:44.537678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 520/1020] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:15:44 2022] POST /identity/v3/services => generated 242 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:44.898765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef440097-42c8-4789-b13e-4aa4f664f5a4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:44.899104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef440097-42c8-4789-b13e-4aa4f664f5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:44.899312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef440097-42c8-4789-b13e-4aa4f664f5a4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:44.900765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 501/1021] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:44.905593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960072b-36de-42d3-a290-dcdcaeb39d85 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:44.905877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960072b-36de-42d3-a290-dcdcaeb39d85 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:44.906146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960072b-36de-42d3-a290-dcdcaeb39d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:44.925413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5960072b-36de-42d3-a290-dcdcaeb39d85 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:44.926603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5960072b-36de-42d3-a290-dcdcaeb39d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:45.005075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 521/1022] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:45.015501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dab76-fefd-4ac4-95d1-8e0beb09f4f0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.015929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dab76-fefd-4ac4-95d1-8e0beb09f4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.016029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303dab76-fefd-4ac4-95d1-8e0beb09f4f0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.018315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 502/1023] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:45.023104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.023427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.023705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.043341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:45.044553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:45.106269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:45.115313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 522/1024] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:45.275804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dab76-fefd-4ac4-95d1-8e0beb09f4f0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.360205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:45.380781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303dab76-fefd-4ac4-95d1-8e0beb09f4f0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.390902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.415949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.416818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.417466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.417735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.428956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:45.433991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:45.434910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:45.436126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 503/1025] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:45.441310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.499458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63fe5e-408e-4cb5-a4b6-7e5b830a0d6e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.535953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.536586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.536861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.537129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.538230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:45.539678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:45.543601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 523/1026] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:45.546812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.556947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82b375-eac5-4ffa-8d04-f38b22c5bad9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.594814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.595616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.596392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.596660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.597465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:45.597896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:45.599411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:45.608193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 504/1027] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:45.613277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:45.613944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.616850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:45.654106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705a069-7c8c-4ea1-92ed-863d63879c49 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.692751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.693347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.693620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.693886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.694530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:45.695983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:45.723634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897d9b-7469-48c4-8304-db6e42527c83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.756785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.757346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.757622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.757890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.768954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:45.770397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:45.784910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:45.785855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:45.792005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:45.792005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:45.792005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:45.792005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 505/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:45.792005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:45.794682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 524/1029] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:15:45 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:15:45.863085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:45.912835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09789a9-cad6-42ba-a2dd-59fed8d6b8d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:45.952362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:45.952970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:45.953248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:45.953515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:45.954209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:45.954452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:45.956160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:45.968335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 506/1030] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:45.975067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.094382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5a4b-962f-4ffa-9f31-7c91057e64b0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.113060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:46.130661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:46.131326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.131610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.131881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.139482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:46.140930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:46.152180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:46.153313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:46.154462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:46.157156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 525/1031] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:46.163084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.272159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab1a10-9b8c-4fba-af96-efe117804e93 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.288192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:46.308817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:46.309581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.309963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.310346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.311201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:46.311609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:46.313433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734eee87-894a-41ec-8c13-ac606d0da540 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:46.332212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 507/1032] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:46.338050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.441019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364d2b9-3c4d-496a-9aed-dce6b52c09cf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.474258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:46.474855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.475129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.475422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.493409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd5c5-0fe8-4b5e-b687-8bc7ce820d4d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.493693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd5c5-0fe8-4b5e-b687-8bc7ce820d4d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.493968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fd5c5-0fe8-4b5e-b687-8bc7ce820d4d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.495574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 508/1033] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:46.500986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.501586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.502629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.526307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:46.527390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:46.529248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:46.549555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:46.551013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:46.603929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 509/1034] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:46.664561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 526/1035] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:46.670041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.771102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c77884-8033-449d-be08-51b7ab3e497e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.772645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:46.805851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:46.806475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.806749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.807014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.817434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:46.818872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:46.833323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:46.834285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:46.835193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:46.835193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:46.835193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:46.842127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 510/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:46.842272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.879618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:46.923011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:46.946036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb3f21-03cf-42e5-a53c-058d344ddd04 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.984629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da432ff-7fd9-4c29-9c70-a8be50d7697b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:46.986749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:46.987482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:46.987881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:46.988029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:46.988656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:46.989038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:46.991263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.009892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 527/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:47.012362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.032779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.033510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.033828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.034166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.034683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.036086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:47.045411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:47.046564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:47.047594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 511/1038] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 16:15:46 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:47.052979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.072826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:47.123641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad8385-e4c1-43b0-905b-ff7123b5f78f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.162496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83943cdf-b343-425d-bf89-7a656b93e644 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.175296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.176083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.176489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.176867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.184394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.185957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.188072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:47.199877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:47.203234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:47.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 528/1039] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:47.208138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.216400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.217090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.217442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.218250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.218962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:47.219357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.221016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.232992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 512/1040] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:47.234442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.335934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ccca5-7d67-414c-a35d-11faadf1004a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.356736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:47.381568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aac3cf-a869-488a-8640-f2b391779320 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.385490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.386243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.386588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.386927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.387738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:47.388113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.389630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.404492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 529/1041] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:47.414007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.431700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.432321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.432593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.432861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.433497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:47.433796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.435190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.441422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 513/1042] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:47.447590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.488537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:47.538999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334e893-1af8-437d-83ac-f028619d0ca9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.559106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81347a91-5889-4130-849e-e8dd801e86c3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.591020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.591895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.592095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.592525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.593280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] RBAC: Adding query filter params (role.id=4726495c16694f73a5adbe329cb1586c, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:47.593792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.596361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.600417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.601262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.601967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.607304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.607467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 530/1043] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=4726495c16694f73a5adbe329cb1586c => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:47.607555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] RBAC: Adding query filter params (interface=public, service_id=b7f5d217ae704eed9606a8a865104b0e, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:47.607555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.607555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2933e8-520b-4f50-beef-004a3d934d8c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.621812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 514/1044] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/endpoints?service_id=b7f5d217ae704eed9606a8a865104b0e&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:47.623099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:47.765953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f779-1397-4fed-bc24-079d40ac2f80 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:47.799763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:47.814338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:47.815070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.815389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.815637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.816237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:47.817599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5111482-307d-4f4d-97af-ce8da61912ed None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:47.831250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 531/1045] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:15:47 2022] GET /identity/v3/services => generated 1957 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:15:47.832383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b8a08-8c9b-4ded-b493-ee061be4db4e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.832666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b8a08-8c9b-4ded-b493-ee061be4db4e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.832944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b8a08-8c9b-4ded-b493-ee061be4db4e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.834529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 515/1046] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:47.842219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:47.844364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:47.846613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:47.867059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:47.868345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:47.949691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 532/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:48.226499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b8a08-8c9b-4ded-b493-ee061be4db4e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:48.239586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:48.338773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b8a08-8c9b-4ded-b493-ee061be4db4e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:48.377068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:48.377821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:48.378135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:48.378403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:48.389290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:48.394453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:48.399319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:48.400466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 516/1048] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:48.401649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:48.521024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e3161-89cd-46e9-907a-7bba05ce4856 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:48.537973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:48.556886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:48.557494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:48.557776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:48.558048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:48.558747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:48.559052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:48.560445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:48.573951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 533/1049] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:48.575882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:48.649146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:48.680581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763c4b-23d2-4c2a-82e8-34da5399ec1d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:48.715652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:48.716304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:48.716579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:48.716847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] PATH_INFO: `/v3/users/e95fc69bbc7f40cca1e9f93063a40848` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:48.731029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] RBAC: Authorizing `identity:get_user(user_id=e95fc69bbc7f40cca1e9f93063a40848)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:48.732511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:48.734322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 517/1050] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:15:48 2022] GET /identity/v3/users/e95fc69bbc7f40cca1e9f93063a40848 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:48.743455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:48.843889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773949ed-8488-4a39-8837-d6aa99807b99 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:48.877683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:48.878361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:48.878642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:48.878915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:48.924996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:48.947495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:48.948951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:49.051189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 534/1051] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:49.057186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:49.157554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abf721-b927-4217-92b3-66eb5c886b7c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:49.184766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:49.193514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:49.194154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.201769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.201769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.201769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:49.201769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d04a741-754f-42fd-8d68-2de3d9e3b846 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:49.201769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 518/1052] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:15:49 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:49.204984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:49.291435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:49.313613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519abf9-b3dc-4ddb-baaf-aed6a92cfc0e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:49.350573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:49.351199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.351482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.351748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.352485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b, user.id=e95fc69bbc7f40cca1e9f93063a40848) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:49.352795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:49.354221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d04623-f555-42ed-a34d-c68e5559a902 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:49.367435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 535/1053] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:49 2022] GET /identity/v3/role_assignments?user.id=e95fc69bbc7f40cca1e9f93063a40848&scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:49.595287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e66ff-3891-41cf-90e5-604a7521ed50 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.596396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e66ff-3891-41cf-90e5-604a7521ed50 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.596396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e66ff-3891-41cf-90e5-604a7521ed50 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.600295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 519/1054] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:49.603124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b21a0-d45f-4cdd-93fb-b6a6c98cd361 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.603552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b21a0-d45f-4cdd-93fb-b6a6c98cd361 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.603817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b21a0-d45f-4cdd-93fb-b6a6c98cd361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.622031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8b6b21a0-d45f-4cdd-93fb-b6a6c98cd361 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:49.623253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b6b21a0-d45f-4cdd-93fb-b6a6c98cd361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:49.691852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 536/1055] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:49.883854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f504-4e6d-4fcd-ae81-2df0b2e86dde None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.884145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f504-4e6d-4fcd-ae81-2df0b2e86dde None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.884318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f504-4e6d-4fcd-ae81-2df0b2e86dde None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.885789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 520/1056] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:49.890708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:49.890997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:49.891302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:49.909715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:49.910900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:49.961750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f504-4e6d-4fcd-ae81-2df0b2e86dde None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:49.978046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 537/1057] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:50.068839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f504-4e6d-4fcd-ae81-2df0b2e86dde None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.102934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.103871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.104391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.104888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.116030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:50.121093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:50.122116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:50.123385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:50.127340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 521/1058] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:50.130042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.132838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.231380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c949b-ddca-41dc-b01a-f39d8e91e379 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.247858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.272692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.273301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.273576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.273844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.274509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:50.274921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.276321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.290429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 538/1059] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:50.291929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.320176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.363794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.365467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd77e6-a246-4876-abee-89da2f37865e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.402640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.403297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.403577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.403843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.404163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde1022-bdbe-4004-804c-9691ff518a2c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.404418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.405786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:50.416042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:50.417124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:50.417965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 522/1060] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:50.423568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.446687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.447348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.447622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.447888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.459545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.460982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:50.475824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:50.476790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:50.477675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:50.477675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:50.477675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:50.477675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 539/1061] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:50.482551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.530437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91054505-afea-4cb1-bec2-f105c9441473 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.551636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.566172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.566896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.567192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.567465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.568083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:50.568378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.569786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.574571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 523/1062] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:50.580209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.592586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41905b-a1f2-4235-9269-757bb2083a32 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.610026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.629070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.629682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.629959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.630241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.630882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:50.631216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.632853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.650194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 540/1063] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:50.651255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.698763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d0a7c-88d8-4d9a-a5a5-06fd9cb3f981 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.735958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.736681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.737174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.737890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.738525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:50.738826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.740601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.750434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 524/1064] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:50.751523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.770000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53484a-fc4e-422c-8681-bf769a8871af None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.808638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.809266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.809548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.809804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.811865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.817409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.818828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:50.830834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:50.831959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:50.832919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:50.835432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 541/1065] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:50.838164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:50.868576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591bc92-d38a-48ba-bb96-dc8f16c1cfe5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.873455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:50.905039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.905638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.905921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.906192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.906861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.908307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.940355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398a4e0-6a31-4a0f-8b62-65627565c1c6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:50.975038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:50.975669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:50.975945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:50.976210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:50.976939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:50.977238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:50.978657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:50.985259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 525/1066] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:15:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:15:50.997500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 542/1067] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:51.003725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.117475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:51.123187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98680025-f311-44cc-86c8-9ecaa25935a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.161998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.162636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.162915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.163296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.173645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:51.175452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ccbdba5f-4b1a-4742-a67e-255878534115 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:51.192723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:51.193742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:51.194691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:51.194691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:51.194691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:51.194691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 526/1068] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:51.199820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.274545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80559-4363-4508-8e6c-428d3baf49c1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.274837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80559-4363-4508-8e6c-428d3baf49c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.275110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d80559-4363-4508-8e6c-428d3baf49c1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.276482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 527/1069] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:51.281361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.281641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.281910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.301222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:51.302584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:51.305404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:51.313157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d678f-31ec-4613-be6d-e70ab4da342b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.351133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.351763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.352035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.352299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.352945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:51.353238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:51.354593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:51.371534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 543/1070] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:51.372896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.386174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 528/1071] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:51.511456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd702bec-4f19-4896-a8bb-1b109f8580b8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.545428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.545974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.546246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.546518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.554032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:51.555736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:51.568341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:51.569267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:51.575295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 544/1072] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:51.575817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.658992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:51.678455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184937b-f54d-4f48-9f56-cec6553f0d8e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.690298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.706872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:51.716014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.716622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.716893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.717160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.717844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:51.718143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:51.719610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:51.731817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 529/1073] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:51.738191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.803042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e846-2290-4c4b-bc32-c7900eafee9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.838109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.838732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.839011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.839307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.842793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8fba-38a7-443d-9af8-5c97cd7e9b6d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:51.850112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:51.854845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:51.855828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:51.856712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 545/1074] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:51.862501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:51.885623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521054ae-8230-4001-841d-af9066379be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:51.886579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521054ae-8230-4001-841d-af9066379be1 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:51.887022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521054ae-8230-4001-841d-af9066379be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:51.887466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521054ae-8230-4001-841d-af9066379be1 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/4726495c16694f73a5adbe329cb1586c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:51.889889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521054ae-8230-4001-841d-af9066379be1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=68764b02b584408090a0c12a06b50445, role_id=4726495c16694f73a5adbe329cb1586c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:51.891462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521054ae-8230-4001-841d-af9066379be1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:51.944632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:51.958571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 530/1075] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:51 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/68764b02b584408090a0c12a06b50445/roles/4726495c16694f73a5adbe329cb1586c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:51.980243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37e5b9-6536-414b-8de6-2eabea8a1d30 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:52.019996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:52.020625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.020906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.021185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.021871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:52.022177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:52.023644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:52.037679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 546/1076] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:52.039053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521054ae-8230-4001-841d-af9066379be1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:52.042038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:52.155064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521054ae-8230-4001-841d-af9066379be1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:52.189489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:52.190626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.190903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.191253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] PATH_INFO: `/v3/users/e95fc69bbc7f40cca1e9f93063a40848` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.192385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] RBAC: Authorizing `identity:get_user(user_id=e95fc69bbc7f40cca1e9f93063a40848)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:52.193843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:52.195752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 531/1077] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:15:52 2022] GET /identity/v3/users/e95fc69bbc7f40cca1e9f93063a40848 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:52.201414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:52.308559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64ada1-3769-4f14-aa99-e94a0b58a8bd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:52.343463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:52.344084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.344360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.344631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.345775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:52.347430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cd859-485e-4761-a015-b65b1e5e4cec None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:52.349255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 547/1078] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:15:52 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:52.354963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:52.432539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:52.464764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9dff3-039d-427a-b0ae-228834672df7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:52.499365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:52.500026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.500307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.500582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b/users/e95fc69bbc7f40cca1e9f93063a40848/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.503084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7f2635629f41fcb7c41ce8bf25005b, user_id=e95fc69bbc7f40cca1e9f93063a40848, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:52.504555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d9b698-6911-4e9f-a2b7-1512663c73cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:52.577312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 532/1079] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:15:52 2022] PUT /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b/users/e95fc69bbc7f40cca1e9f93063a40848/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:15:52.967648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff89f8d-1ad9-4c12-a4cd-43a909367c78 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.968200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff89f8d-1ad9-4c12-a4cd-43a909367c78 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.968200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff89f8d-1ad9-4c12-a4cd-43a909367c78 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.969640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 548/1080] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:52.975310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:52.975729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:52.975906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:52.995211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:52.996320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:53.067089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 533/1081] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:53.333816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff89f8d-1ad9-4c12-a4cd-43a909367c78 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:53.337762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:53.437072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff89f8d-1ad9-4c12-a4cd-43a909367c78 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:53.472697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:53.473304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:53.473580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:53.473854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:53.541435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:53.542894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:53.545616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:53.669307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 549/1082] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:15:53.675876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:53.742845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:53.795676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024eef7-b2a2-45ae-ba50-984c28ead27b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:53.832924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:53.833740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:53.834110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:53.834475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:53.844811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:53.846716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c778a7d6-7857-46f2-8956-68a0aba3c68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:53.863675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:53.864861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:53.865990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:53.865990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:53.865990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:53.871179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 534/1083] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:53.873225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:53.893119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ccb06-7378-47b9-ae32-05fc8a8fbebb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:53.893528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ccb06-7378-47b9-ae32-05fc8a8fbebb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:53.893978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ccb06-7378-47b9-ae32-05fc8a8fbebb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:53.895346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 535/1084] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:53.900950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:53.901231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:53.901501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:53.920903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:53.922163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:53.983095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49616f46-ae85-41d7-9796-b193ba139ea1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:53.986186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:53.996217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 536/1085] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:54.018919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:54.019630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.019911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.020183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.021216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:54.021880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:54.023769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:54.035579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 550/1086] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:54.267406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.312930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:54.369658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19092571-1f1b-4fb9-a979-7e030037451a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:54.402307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:54.402935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.403245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.403519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.414074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bf9cee68-1036-4856-8660-9edc1ab82ef7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:54.418869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:54.419937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:54.421242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 537/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:54.426705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.526160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c91304-58ed-45ad-9626-c3658c7fbba7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:54.560714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:54.561323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.561605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.562053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.562713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:54.563023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:54.563575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975757f-85b4-40b6-bb95-13e81fa43a99 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.563860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975757f-85b4-40b6-bb95-13e81fa43a99 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.564135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975757f-85b4-40b6-bb95-13e81fa43a99 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.564416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:54.566297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 538/1088] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:15:54.571077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.571378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.571649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.578035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 551/1089] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:54.579001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.594179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:54.595401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:54.607376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:54.664146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 539/1090] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:15:54.679202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306accd4-9c39-4282-9434-2e9a25ca92ca None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:54.714224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efee15-0081-4228-85c7-71f662df114b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:54.714841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efee15-0081-4228-85c7-71f662df114b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.715119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efee15-0081-4228-85c7-71f662df114b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.715437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efee15-0081-4228-85c7-71f662df114b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.726213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efee15-0081-4228-85c7-71f662df114b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:54.727711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efee15-0081-4228-85c7-71f662df114b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-18efee15-0081-4228-85c7-71f662df114b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:54.742388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:54.743525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:54.744875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:54.744875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:54.744875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:54.744875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 552/1091] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:54.749376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.779896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:54.847243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841f58e-eda7-4ce2-a8f2-04651fcaf930 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:54.880964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:54.881570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:54.881846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:54.882116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:54.882757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:54.883055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:54.884464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:54.895722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 540/1092] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:54.901241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efee15-0081-4228-85c7-71f662df114b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.939004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:54.980549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.003029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efee15-0081-4228-85c7-71f662df114b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.029533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.039645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.039645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.039645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.039942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795b5bf-416c-439c-85f1-6284e42ef567 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.040050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.047355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.048794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:15:55.060834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:55.062000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:55.063172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 553/1093] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:55.067781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.075781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.076413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.076688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.076954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.086929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:15:55.091963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:55.092931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:15:55.093766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 541/1094] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:55.099628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.166108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bed06-ad3e-43d9-8cfc-379bfc3646a4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.199369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc572ac-5635-40d7-a4a9-1ef727b51344 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.201161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.201766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.202042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.202312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.203039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:55.203370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.204803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.213147 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.222434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 554/1095] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:55.228097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.234003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.234610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.234894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.235173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.235860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:55.236160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.238213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.246880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 542/1096] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:55.252610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.268721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.337313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18d842-3bbd-4827-bcd2-d697990d879c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.351179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db56a8-0ffc-4cb8-8485-41523fbbd910 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.370774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.371405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.371686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.371957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.384692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.385317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.385597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.385872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] PATH_INFO: `/v3/users/e95fc69bbc7f40cca1e9f93063a40848` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.386982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] RBAC: Authorizing `identity:get_user(user_id=e95fc69bbc7f40cca1e9f93063a40848)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.388449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.390017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 543/1097] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/users/e95fc69bbc7f40cca1e9f93063a40848 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:55.396335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.437640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.444571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.446016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.479729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.497346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289b5bb-e008-4dbb-b6f0-072736184424 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.530390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.530989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.531291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.531560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.544569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 555/1098] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:15:55.550806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.617836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.617836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.645397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.650929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831da9f-2029-4819-aef0-4eb633b83f5f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.680208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.686557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.687202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.687483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.687759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.698369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.699600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 544/1099] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:15:55.699834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.705709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:15:55.714699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:55.715787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:15:55.716689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:15:55.716689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:15:55.716689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:55.716689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 556/1100] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:15:55.721936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.805359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201e8e8-8d2d-4132-b802-216c162db9d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.824274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c47493-0908-4052-a996-c6d65b347deb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.841551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.842184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.842465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.842740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.843925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.845354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.847703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 545/1101] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:55.853802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.862361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.862965 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.863277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.863552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.864195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:55.864498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.865883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.877996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 557/1102] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:55.883895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:55.894636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.903750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:55.953386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c55ed8-f20d-49b1-932b-8ad1d1c4aa8f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.983421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcc098-27da-459e-9945-625b707a4c66 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:55.988021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:55.988628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:55.988907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:55.989176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:55.989902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b, user.id=e95fc69bbc7f40cca1e9f93063a40848) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:55.990205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:55.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:55.998561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 546/1103] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/role_assignments?user.id=e95fc69bbc7f40cca1e9f93063a40848&scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:15:56.018207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:56.018857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:56.019128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:56.019511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:56.027133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:56.028696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:15:56.042344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:15:56.043366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:15:56.044267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 558/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:15:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:15:56.051440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:56.160121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d8e1d-d4b4-4f2f-8504-6a98bee6db05 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:56.173983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:56.194029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:56.194742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:56.195028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:56.195331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:56.196024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:56.196326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:56.197899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7545b-43c5-4b09-86b2-6e8e76adf4b4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:56.211435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 547/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:15:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:56.217939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:56.285497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:56.319273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb888a8-9d68-4414-a762-1aefc12019bc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:56.353224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:56.353820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:56.354096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:56.354491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:56.355892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] RBAC: Adding query filter params (role.id=4726495c16694f73a5adbe329cb1586c, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=68764b02b584408090a0c12a06b50445) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:15:56.356457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:56.358209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b607f5-41f2-487e-bece-85f7ef3198a6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:56.364149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 559/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:15:56 2022] GET /identity/v3/role_assignments?user.id=68764b02b584408090a0c12a06b50445&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=4726495c16694f73a5adbe329cb1586c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:57.999515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39a58-8138-4ffa-b5b7-2e49045b81a5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:58.000049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39a58-8138-4ffa-b5b7-2e49045b81a5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:58.000049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a39a58-8138-4ffa-b5b7-2e49045b81a5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:58.001394 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 548/1107] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:15:58.007304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:58.007735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:58.007854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:58.026885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:15:58.028094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:15:58.092558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 560/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:15:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:15:58.373069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a39a58-8138-4ffa-b5b7-2e49045b81a5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:58.471526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a39a58-8138-4ffa-b5b7-2e49045b81a5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:58.505485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:58.506097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:58.506380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:58.506651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:58.507758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:58.509184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c951a-5c40-4d59-bd78-6a8a8a49d55d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:58.510707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 549/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:15:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:15:58.516441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:15:58.576046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:15:58.613117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabebe00-d919-4909-b01a-b2df10139c65 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:15:58.645912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:15:58.646446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:15:58.646720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:15:58.646989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:15:58.647672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:15:58.649086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8daf63-4879-44a4-a510-0f9d1a5dce44 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:15:58.770525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 561/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:15:58 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:00.727442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbff36-414e-48bc-a0fe-30e76b84643b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:00.727970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbff36-414e-48bc-a0fe-30e76b84643b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:00.727970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbff36-414e-48bc-a0fe-30e76b84643b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:00.729315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 550/1111] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:00.734389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:00.734818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:00.734919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:00.754331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:00.755550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:00.820757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 562/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:16:01.094660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbff36-414e-48bc-a0fe-30e76b84643b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:01.139759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:01.198087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbff36-414e-48bc-a0fe-30e76b84643b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:01.231996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:01.232614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:01.232888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:01.233154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] PATH_INFO: `/v3/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:01.234517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] RBAC: Authorizing `identity:get_role(role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:01.235988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:01.237857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 551/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:01 2022] GET /identity/v3/roles/8488c378257a4f629184ae9773b882ac => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:01.243805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:01.290443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:01.340337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd981e45-67e8-4516-8bbc-38dd44fdeb88 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:01.373888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:01.374448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:01.374727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:01.375003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] PATH_INFO: `/v3/users/f4a84366906642ea87442d2cb4065ef0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:01.387711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] RBAC: Authorizing `identity:get_user(user_id=f4a84366906642ea87442d2cb4065ef0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:01.389144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:01.390726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 563/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:01 2022] GET /identity/v3/users/f4a84366906642ea87442d2cb4065ef0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:01.397389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:01.504293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24360d06-064d-4a36-9df9-0e055cf20c63 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:01.517202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:01.541622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:01.542221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:01.542501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:01.542773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:01.608562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:01.610018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:01.704733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 552/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:16:01.711110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:01.808729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d49b8-fb48-4b0a-b66e-dbaebb8b6f7a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:01.815837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:01.841463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:01.842082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:01.842359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:01.842630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:01.843746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:01.845168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09e42e-1ee9-4ffa-8c39-b4ea14c0831f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:01.847437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 564/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:01 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:01.853639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:01.865612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:01.951200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee95fe6-71a5-4d52-b7e9-e13e6dcf6c30 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:01.984562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:01.985102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:01.985381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:01.985650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:01.986400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b, user.id=f4a84366906642ea87442d2cb4065ef0) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:01.986707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:01.988872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20726a30-5342-407d-bafe-adc40846b211 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:01.994086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 553/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:01 2022] GET /identity/v3/role_assignments?user.id=f4a84366906642ea87442d2cb4065ef0&scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b&role.id=8488c378257a4f629184ae9773b882ac => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:03.876927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6f7bf-3915-43b8-8bc4-d6fde2143a90 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:03.877493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6f7bf-3915-43b8-8bc4-d6fde2143a90 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:03.877493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6f7bf-3915-43b8-8bc4-d6fde2143a90 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:03.879612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 565/1118] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:03.884803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:03.885213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:03.885355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:03.904523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:03.905719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:03.972396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 554/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:04.242045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6f7bf-3915-43b8-8bc4-d6fde2143a90 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:04.342021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6f7bf-3915-43b8-8bc4-d6fde2143a90 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:04.374785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:04.375424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:04.375701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:04.375971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] PATH_INFO: `/v3/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:04.377351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] RBAC: Authorizing `identity:get_role(role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:04.379324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:04.380617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 566/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:04 2022] GET /identity/v3/roles/8488c378257a4f629184ae9773b882ac => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:04.386404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:04.389484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:04.484129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47e7ac-ef03-46a6-893a-e3fbaff2a8b7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:04.517266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:04.517828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:04.518103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:04.518406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] PATH_INFO: `/v3/users/f4a84366906642ea87442d2cb4065ef0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:04.519620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] RBAC: Authorizing `identity:get_user(user_id=f4a84366906642ea87442d2cb4065ef0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:04.521055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:04.522616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 555/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:04 2022] GET /identity/v3/users/f4a84366906642ea87442d2cb4065ef0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:04.528411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:04.571579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:04.625405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7426ab43-ebd6-48c9-bb0b-9a2fe0ba8f56 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:04.658013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:04.658622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:04.659434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:04.659802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:04.660912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:04.662324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904e54-08a2-4a67-8e14-2d2733392f34 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:04.663940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 567/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:04 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:04.669868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:04.738719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:04.767069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab28d65-8f32-4048-9a33-a8ca25ffabbc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:04.799268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:04.800468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:04.800742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:04.801013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b/users/f4a84366906642ea87442d2cb4065ef0/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:04.803136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc7f2635629f41fcb7c41ce8bf25005b, user_id=f4a84366906642ea87442d2cb4065ef0, role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:04.804593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec521a8-0097-4449-a0f2-5313eba2587d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:04.861723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 556/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:16:04 2022] PUT /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b/users/f4a84366906642ea87442d2cb4065ef0/roles/8488c378257a4f629184ae9773b882ac => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:16:06.764470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0f39-c047-4a27-832e-59b8d9a0bb16 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:06.765026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0f39-c047-4a27-832e-59b8d9a0bb16 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:06.765026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e0f39-c047-4a27-832e-59b8d9a0bb16 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:06.766316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 568/1124] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:06.771664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:06.772040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:06.772210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:06.790180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:06.791395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:06.855869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 557/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:07.137033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e0f39-c047-4a27-832e-59b8d9a0bb16 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:07.237402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e0f39-c047-4a27-832e-59b8d9a0bb16 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:07.249159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:07.272852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] PATH_INFO: `/v3/roles/8488c378257a4f629184ae9773b882ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] RBAC: Authorizing `identity:get_role(role_id=8488c378257a4f629184ae9773b882ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:07.280111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 569/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:07 2022] GET /identity/v3/roles/8488c378257a4f629184ae9773b882ac => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:07.283781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:07.343133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:07.380856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a658294-5344-490d-8238-8b3f34292f3d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:07.413536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:07.414102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:07.414381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:07.414656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] PATH_INFO: `/v3/users/f4a84366906642ea87442d2cb4065ef0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:07.415784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] RBAC: Authorizing `identity:get_user(user_id=f4a84366906642ea87442d2cb4065ef0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:07.417223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:07.418787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 558/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:07 2022] GET /identity/v3/users/f4a84366906642ea87442d2cb4065ef0 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:07.424503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:07.522438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93b9e10-4cf8-45c7-9c76-e02e96a7db88 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:07.555693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:07.556361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:07.556892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:07.557164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:07.603310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:07.624782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:07.626215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:07.721497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 570/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:16:07.728039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:07.824850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4c51b-3e00-40c1-b54d-5aede075fd33 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:07.852594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:07.858167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:07.863438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:07.863438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:07.863438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] PATH_INFO: `/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:07.863438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] RBAC: Authorizing `identity:get_project(project_id=dc7f2635629f41fcb7c41ce8bf25005b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:07.863438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbcaf6-a996-4d4c-9e21-0b59fc12ef41 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:07.864519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 559/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:07 2022] GET /identity/v3/projects/dc7f2635629f41fcb7c41ce8bf25005b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:07.870631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:07.947746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:07.968112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdb03c-74c4-4091-afea-95a1f5d0e32b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:08.002405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:08.003030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:08.003421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:08.003698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:08.004971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] RBAC: Adding query filter params (role.id=8488c378257a4f629184ae9773b882ac, scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b, user.id=f4a84366906642ea87442d2cb4065ef0) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:08.004971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:08.006123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e59d1-2343-4b52-93ae-8a4455be1f71 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:08.011679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 571/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:07 2022] GET /identity/v3/role_assignments?user.id=f4a84366906642ea87442d2cb4065ef0&scope.project.id=dc7f2635629f41fcb7c41ce8bf25005b&role.id=8488c378257a4f629184ae9773b882ac => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:09.927962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1c85-91b1-4df9-919e-c1f577b46d28 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:09.928527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1c85-91b1-4df9-919e-c1f577b46d28 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:09.928527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f1c85-91b1-4df9-919e-c1f577b46d28 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:09.929837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 560/1131] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:09.935132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:09.935565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:09.935711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:09.954138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:09.955347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:10.019778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 572/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:10.290020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1c85-91b1-4df9-919e-c1f577b46d28 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:10.391273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f1c85-91b1-4df9-919e-c1f577b46d28 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:10.423520 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:10.424132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:10.424408 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:10.424673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:10.426533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:10.427981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b654fcb-e4de-4847-b317-0414253e8e08 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:10.429527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 561/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:10.435581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:10.504924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:10.532748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eab963-2832-4ebd-a70f-5470fc403640 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:10.567552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:10.568146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:10.568420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:10.568682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:10.569319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:10.570730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe167f5-7842-49c8-b8dd-8187482d7670 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:10.652934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 573/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:16:10 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:12.590067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146eaf6-517b-475f-9d4e-46b9dcb65916 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:12.590591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146eaf6-517b-475f-9d4e-46b9dcb65916 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:12.590591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146eaf6-517b-475f-9d4e-46b9dcb65916 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:12.591968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 562/1135] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:12.597375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:12.603329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:12.603329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:12.617290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:12.618480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:12.683875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 574/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 01 16:16:12.974176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146eaf6-517b-475f-9d4e-46b9dcb65916 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:13.039355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:13.074487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146eaf6-517b-475f-9d4e-46b9dcb65916 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:13.107424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:13.108045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:13.108326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:13.108597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:13.109705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:13.111139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5b3f0-9459-4651-ba40-c3d51d84a7bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:13.112770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 563/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:16:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:13.118536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:13.172347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:13.215377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68b9ec-a3d3-437e-88f4-d18752bfc9d0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:13.248641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60264b-361a-441b-8570-b5232050822d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:13.249270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60264b-361a-441b-8570-b5232050822d None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:13.249549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60264b-361a-441b-8570-b5232050822d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:13.249818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60264b-361a-441b-8570-b5232050822d None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:13.250481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60264b-361a-441b-8570-b5232050822d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:13.251959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60264b-361a-441b-8570-b5232050822d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:13.321160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 575/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:16:13 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:15.258657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135ba0-2a50-4589-bdb1-a9625887c1e0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:15.259115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135ba0-2a50-4589-bdb1-a9625887c1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:15.259259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a135ba0-2a50-4589-bdb1-a9625887c1e0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:15.260546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 564/1139] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:15.265756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:15.266193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:15.266295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:15.284632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:15.285902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:15.350350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 576/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:15.623013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135ba0-2a50-4589-bdb1-a9625887c1e0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:15.725963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a135ba0-2a50-4589-bdb1-a9625887c1e0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:15.758940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:15.759592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:15.759871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:15.760142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:15.762000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:15.771464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:15.777991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:15.779271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:15.780369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 565/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:15.783582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:15.824306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:15.882388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ebce6-077c-4678-a7d5-fe36d3e9feed None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:15.916066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:15.916658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:15.916931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:15.917197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:15.917845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:15.918139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:15.919520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:15.927376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 577/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:15.933514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:16.031926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c72d0b-b7b4-4c65-8fa0-a812dc7db409 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:16.064522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:16.065147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:16.065421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:16.065690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] PATH_INFO: `/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:16.079176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d5b7ca77e74dd7aeb9487f253e6f40)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:16.080635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:16.083084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 566/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:15 2022] GET /identity/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:16.088999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:16.184771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5614f4-15a8-459a-a4d2-9e9cb888d19f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:16.186190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:16.218167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:16.218782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:16.219059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:16.219354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:16.286928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:16.288399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:16.384809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 578/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:16:16.390714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:16.443005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:16.489960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cbd3e-5b58-4340-920e-7538eff3ab75 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:16.523056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] PATH_INFO: `/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] RBAC: Authorizing `identity:get_project(project_id=986a0e2d266a41f8aecf11adc8bec4c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9ebab-2990-4326-9852-f748abaa817f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:16.535855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 567/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:16 2022] GET /identity/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:16.536387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:16.536957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:16.635812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23ef93-5604-40c8-a335-641c3a721fde None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:16.668941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:16.669530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:16.669804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:16.670071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:16.670789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=986a0e2d266a41f8aecf11adc8bec4c5, user.id=f3d5b7ca77e74dd7aeb9487f253e6f40) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:16.671096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:16.672539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54e2cd-4bee-433c-b242-517e7e50afe8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:16.678660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 579/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:16 2022] GET /identity/v3/role_assignments?user.id=f3d5b7ca77e74dd7aeb9487f253e6f40&scope.project.id=986a0e2d266a41f8aecf11adc8bec4c5&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:18.560030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4793a-294c-47a6-a7bd-dfb5ac1856bf None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:18.560576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4793a-294c-47a6-a7bd-dfb5ac1856bf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:18.560576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4793a-294c-47a6-a7bd-dfb5ac1856bf None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:18.561890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 568/1147] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:18.567119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:18.567543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:18.567696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:18.586418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:18.587626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:18.645641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:18.652852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 580/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:18.928796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4793a-294c-47a6-a7bd-dfb5ac1856bf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:19.029639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4793a-294c-47a6-a7bd-dfb5ac1856bf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:19.049099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:19.063227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:19.063849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:19.064122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:19.064389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:19.074912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:19.079863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:19.080933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:19.082942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 569/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:19.087625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:19.182149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8195d97-8281-4d3f-90fd-fdfa3d64368e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:19.215646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:19.216246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:19.216524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:19.216799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:19.217465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:19.217766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:19.219123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:19.227503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 581/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:19.233206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:19.334661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d7a92-3d5b-4464-b49a-94c04c3c63b4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:19.367913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:19.368535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:19.368812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:19.369086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] PATH_INFO: `/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:19.370194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d5b7ca77e74dd7aeb9487f253e6f40)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:19.371664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:19.373232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 570/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:19 2022] GET /identity/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:19.379117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:19.435813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:19.476632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db158e5-15e0-44f7-98d2-b11456489d30 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:19.508679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:19.509286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:19.509562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:19.509830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] PATH_INFO: `/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:19.510922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] RBAC: Authorizing `identity:get_project(project_id=986a0e2d266a41f8aecf11adc8bec4c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:19.512360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabb672-d8d3-4948-ab42-a52ea76f0283 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:19.513936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 582/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:19 2022] GET /identity/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:19.520743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:19.569203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:19.623268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a0407-ee28-423b-9375-695fde2c9c15 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:19.656404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:19.657054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:19.657329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:19.657598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] PATH_INFO: `/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5/users/f3d5b7ca77e74dd7aeb9487f253e6f40/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:19.659493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] RBAC: Authorizing `identity:create_grant(project_id=986a0e2d266a41f8aecf11adc8bec4c5, user_id=f3d5b7ca77e74dd7aeb9487f253e6f40, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:19.661623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bfd615-0948-4e03-b5ae-4026b23a14ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:19.731479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 571/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:16:19 2022] PUT /identity/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5/users/f3d5b7ca77e74dd7aeb9487f253e6f40/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:16:21.618636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb5bb-afd4-4097-b623-850de83d5afe None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:21.619184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb5bb-afd4-4097-b623-850de83d5afe None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:21.619184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1bb5bb-afd4-4097-b623-850de83d5afe None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:21.620510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 583/1154] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:21.625926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:21.626374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:21.626476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:21.645126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:21.651835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:21.711374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 572/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:21.982702 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bb5bb-afd4-4097-b623-850de83d5afe None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.084149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1bb5bb-afd4-4097-b623-850de83d5afe None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:22.109635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:22.117953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:22.118567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:22.118838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:22.119104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:22.129363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:22.134280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:22.135451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:16:22.136479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 584/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:22.141576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.181019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:22.240251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9766d-dfd7-47c6-adb9-f25075ad5186 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:22.274992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:22.275707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:22.275988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:22.276260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:22.276921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:22.277222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:22.278593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:22.286506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 573/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:22.292540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.390052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876b7af-ee08-46db-a0ac-b0073333aa5e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:22.422121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:22.422735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:22.423010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:22.423310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] PATH_INFO: `/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:22.424416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] RBAC: Authorizing `identity:get_user(user_id=f3d5b7ca77e74dd7aeb9487f253e6f40)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:22.426438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:22.428007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 585/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:22 2022] GET /identity/v3/users/f3d5b7ca77e74dd7aeb9487f253e6f40 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:22.434148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.528995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:22.531846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf4260-9f2d-440d-9daa-de3c7de7142a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:22.565012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:22.565611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:22.565892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:22.566704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:22.632231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:22.633719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:22.726015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:22.729379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 574/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:16:22.735565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.790115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:22.834323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067d8fb-9a87-4559-8da6-26dbbeb7e972 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:22.868173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:22.868785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:22.869059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:22.869406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] PATH_INFO: `/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:22.870507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] RBAC: Authorizing `identity:get_project(project_id=986a0e2d266a41f8aecf11adc8bec4c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:22.871938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f3161-a6b8-4e0e-874d-4536322ea232 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:22.873534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 586/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:22 2022] GET /identity/v3/projects/986a0e2d266a41f8aecf11adc8bec4c5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:22.879792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:22.983717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfa9dd-cae7-45b0-811d-26336a26a3d9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:23.026535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:23.027141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:23.027455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:23.027732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:23.028503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=986a0e2d266a41f8aecf11adc8bec4c5, user.id=f3d5b7ca77e74dd7aeb9487f253e6f40) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:23.028810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:23.030265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae41f27-43e4-4f50-853d-cc4f0eb4c01c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:23.039304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 575/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:22 2022] GET /identity/v3/role_assignments?user.id=f3d5b7ca77e74dd7aeb9487f253e6f40&scope.project.id=986a0e2d266a41f8aecf11adc8bec4c5&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:25.006311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d284574-a616-429a-b2ee-fb4e9ea72b24 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:25.006868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d284574-a616-429a-b2ee-fb4e9ea72b24 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:25.006868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d284574-a616-429a-b2ee-fb4e9ea72b24 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:25.008210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 587/1162] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:25.013267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:25.013690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:25.013812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:25.032316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:25.033501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:25.091571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:25.099529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 576/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:25.367609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d284574-a616-429a-b2ee-fb4e9ea72b24 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:25.465916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d284574-a616-429a-b2ee-fb4e9ea72b24 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:25.487599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:25.499281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:25.499872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:25.500148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:25.500417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:25.564879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:25.566354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:25.661473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 588/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:16:25.667369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:25.762417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4eb74e-57c0-4264-938e-05734f963e55 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:25.796116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:25.796737 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:25.797012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:25.797283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:25.804993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:25.806445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-067607b4-27ba-435e-b9ad-969fdedda1b0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:25.818102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:25.819331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:25.820360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 577/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:16:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:25.825214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:25.848795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:25.921184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74d4bf-33f2-4e81-b81d-9b7fd5bce9a6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:25.954666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:25.955282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:25.955560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:25.955830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:25.956464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:25.956761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:25.958874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17b59e-6cc2-4c73-8b18-2a56ae4a8a80 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:25.965721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 589/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Sep 1 16:16:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:27.846794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccee6-4a31-4080-a593-3543f38ee70d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:27.847545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccee6-4a31-4080-a593-3543f38ee70d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:27.847545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adccee6-4a31-4080-a593-3543f38ee70d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:27.849143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 578/1167] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:27.854193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f31a-0380-4280-b04b-ede23aad10d2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:27.854617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f31a-0380-4280-b04b-ede23aad10d2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:27.854722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f31a-0380-4280-b04b-ede23aad10d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:27.872952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a652f31a-0380-4280-b04b-ede23aad10d2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:27.874122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a652f31a-0380-4280-b04b-ede23aad10d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:27.938725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 590/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:28.206427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adccee6-4a31-4080-a593-3543f38ee70d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:28.262500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:28.307350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adccee6-4a31-4080-a593-3543f38ee70d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:28.340854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:28.341456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:28.341759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:28.342027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:28.342698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:28.344083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1284d-6f38-456c-86a2-8678e35c14b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:28.437063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 579/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Sep 1 16:16:28 2022] POST /identity/v3/domains => generated 267 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:30.445207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738ed77-8c07-44f6-ba39-990b214018ff None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:30.445757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738ed77-8c07-44f6-ba39-990b214018ff None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:30.445757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738ed77-8c07-44f6-ba39-990b214018ff None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:30.447074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 591/1170] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:30.452275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:30.452689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:30.452825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:30.471124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:30.472341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:30.540309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 580/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:30.816021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738ed77-8c07-44f6-ba39-990b214018ff None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:30.834203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:30.915738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738ed77-8c07-44f6-ba39-990b214018ff None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:30.949369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:30.949979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:30.950250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:30.950517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] PATH_INFO: `/v3/domains/33697e6ac77f48da8accaad1234e7a3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:30.960911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=33697e6ac77f48da8accaad1234e7a3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:30.962328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfecc1-c033-4e86-a5d1-9c70bb821c93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:30.963904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 592/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 1 16:16:30 2022] GET /identity/v3/domains/33697e6ac77f48da8accaad1234e7a3d => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:30.969805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:31.013746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:31.068416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabfadc5-b08f-465d-84f0-b2f837d13531 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:31.101855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:31.102453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:31.102726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:31.102990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:31.103657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:31.105073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb6c14-ba72-415e-91e0-22d55f493598 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:31.201508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 581/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:16:30 2022] POST /identity/v3/projects => generated 361 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:33.133001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba836-9c4d-44b1-8ac8-372aab62a728 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:33.133591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba836-9c4d-44b1-8ac8-372aab62a728 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:33.133591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba836-9c4d-44b1-8ac8-372aab62a728 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:33.134870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 593/1174] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:33.140141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:33.140565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:33.140697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:33.159713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:33.161471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:33.226984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 582/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:33.497389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba836-9c4d-44b1-8ac8-372aab62a728 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:33.573562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:33.596316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba836-9c4d-44b1-8ac8-372aab62a728 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:33.629749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:33.630363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:33.630639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:33.630909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] PATH_INFO: `/v3/domains/33697e6ac77f48da8accaad1234e7a3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:33.632041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=33697e6ac77f48da8accaad1234e7a3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:33.633465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62992d-9ee8-4ae3-aed1-7284b6741738 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:33.635025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 594/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Sep 1 16:16:33 2022] GET /identity/v3/domains/33697e6ac77f48da8accaad1234e7a3d => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:33.640784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:33.668153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:33.738649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d04eff-2c3d-4b59-ac3d-5ff4d52062d2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:33.772884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:33.773482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:33.773759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:33.774032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:33.774709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:33.776179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc948e9-ce15-4a23-99f4-ea38d06aea43 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:33.853708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 583/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:16:33 2022] POST /identity/v3/users => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:16:35.858336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcef1e2-dd8f-4429-a1fe-61dcc54a419b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:35.858911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcef1e2-dd8f-4429-a1fe-61dcc54a419b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:35.858911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcef1e2-dd8f-4429-a1fe-61dcc54a419b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:35.860218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 595/1178] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:35.865497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:35.865871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:35.866044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:35.884020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:35.885218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:35.955407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 584/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:36.223499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcef1e2-dd8f-4429-a1fe-61dcc54a419b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:36.333130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcef1e2-dd8f-4429-a1fe-61dcc54a419b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:36.365606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:36.366226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:36.366506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:36.366779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:36.376687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:36.382054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:36.383470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:16:36.384810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 596/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:16:36.389403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:36.404318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:36.489067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcc451-9cb7-41d8-b4e7-5bad0b3cbb60 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:36.523107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:36.523743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:36.524018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:36.524291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:36.524950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:36.525250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:36.526663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:36.534536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 585/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:36.540665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:36.545504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:36.642238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a842576-619b-4261-be3d-b60218b5c55d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:36.723359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:36.723978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:36.724254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:36.724521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] PATH_INFO: `/v3/users/651f54adbe2148c094738bcf934e8a8c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:36.737069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] RBAC: Authorizing `identity:get_user(user_id=651f54adbe2148c094738bcf934e8a8c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:36.739002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:36.740670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 597/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:36 2022] GET /identity/v3/users/651f54adbe2148c094738bcf934e8a8c => generated 311 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:36.746669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:36.820959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:36.844938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236643a6-8c33-458a-a2a6-5a6b5699b7c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:36.878225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:36.879634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:36.879885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:36.880158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:36.946070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:36.947551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:37.018629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:37.042944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 586/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 01 16:16:37.048950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:37.108708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:37.146311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e7c3f-4eb6-4b4a-b26a-d49c3e33f118 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:37.179769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:37.180386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:37.180662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:37.180933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] PATH_INFO: `/v3/projects/64997bc8748642b0bd578e4146842613` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:37.182039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] RBAC: Authorizing `identity:get_project(project_id=64997bc8748642b0bd578e4146842613)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:37.183748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5dad7-e6e8-482c-bfad-4620502233a5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:37.185348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 598/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:37 2022] GET /identity/v3/projects/64997bc8748642b0bd578e4146842613 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:37.192009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:37.292662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdbd20-b041-493a-a1a5-ade7e7f68d8a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:37.327742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:37.328335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:37.328611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:37.328880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:37.329593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=64997bc8748642b0bd578e4146842613, user.id=651f54adbe2148c094738bcf934e8a8c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:37.329896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:37.331403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2a878-6d33-4af2-a4a2-74b90b890f22 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:37.336573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 587/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:37 2022] GET /identity/v3/role_assignments?user.id=651f54adbe2148c094738bcf934e8a8c&scope.project.id=64997bc8748642b0bd578e4146842613&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:39.247013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b88f-7999-45ee-b4c8-555ded18436e None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:39.247593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b88f-7999-45ee-b4c8-555ded18436e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:39.247593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022b88f-7999-45ee-b4c8-555ded18436e None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:39.248915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 599/1186] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:39.254024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:39.254443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:39.254549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:39.272448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:39.273632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:39.303071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:39.338979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 588/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:39.607130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022b88f-7999-45ee-b4c8-555ded18436e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:39.708351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022b88f-7999-45ee-b4c8-555ded18436e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:39.736917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:39.742352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:39.742979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:39.743290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:39.743565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:39.753309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:39.758109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:39.759204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:16:39.760241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 600/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:39.765613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:39.862811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e75db-b3f8-40bd-856f-6205d5cccb92 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:39.896484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:39.897078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:39.897359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:39.897632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:39.898294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:39.898597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:39.900002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:39.907793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 589/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:39.913698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:40.018802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243bd461-8df9-440d-ac18-539b2fad345b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:40.052412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:40.053032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:40.053308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:40.053581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] PATH_INFO: `/v3/users/651f54adbe2148c094738bcf934e8a8c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:40.054694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] RBAC: Authorizing `identity:get_user(user_id=651f54adbe2148c094738bcf934e8a8c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:40.056141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:40.057950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 601/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:39 2022] GET /identity/v3/users/651f54adbe2148c094738bcf934e8a8c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:40.063823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:40.096001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:40.164467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdce621a-e56f-4e3d-8f43-273dfea6a4fd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:40.197991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:40.198610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:40.198885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:40.199177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] PATH_INFO: `/v3/projects/64997bc8748642b0bd578e4146842613` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:40.200278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] RBAC: Authorizing `identity:get_project(project_id=64997bc8748642b0bd578e4146842613)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:40.201709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a82ed-14e4-4461-808c-32b70b2fb801 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:40.203357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 590/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:40 2022] GET /identity/v3/projects/64997bc8748642b0bd578e4146842613 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:40.210053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:40.273142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:40.329537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c618d3-0d93-4fa3-b576-861f3aaa58ce None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:40.371839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:40.372513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:40.372789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:40.373062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] PATH_INFO: `/v3/projects/64997bc8748642b0bd578e4146842613/users/651f54adbe2148c094738bcf934e8a8c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:40.375005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] RBAC: Authorizing `identity:create_grant(project_id=64997bc8748642b0bd578e4146842613, user_id=651f54adbe2148c094738bcf934e8a8c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:40.376500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57bd5a-eb98-4e64-a55c-6bb58a32c8ef None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:40.434974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 602/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:16:40 2022] PUT /identity/v3/projects/64997bc8748642b0bd578e4146842613/users/651f54adbe2148c094738bcf934e8a8c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:16:42.461867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e886f-5ce5-4c83-ad1f-2ea2bba9b9be None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:42.462407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e886f-5ce5-4c83-ad1f-2ea2bba9b9be None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:42.462519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e886f-5ce5-4c83-ad1f-2ea2bba9b9be None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:42.463957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 591/1193] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:16:42.469134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:42.469508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:42.469681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:42.487556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:16:42.488766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:16:42.555106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 603/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:16:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:16:42.825131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e886f-5ce5-4c83-ad1f-2ea2bba9b9be None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:42.919527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:42.924022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e886f-5ce5-4c83-ad1f-2ea2bba9b9be None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:42.957396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:42.958017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:42.958292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:42.958561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:42.968445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:16:42.973907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:16:42.975087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:16:42.975978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 592/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:16:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:16:42.980936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:43.027758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:43.077051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b607213-ea09-4bc2-89ed-b0246d87f17b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:43.109946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:43.110537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:43.110811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:43.111077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:43.111772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:43.112071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:43.114233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:43.121921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 604/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:16:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:43.127703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:43.224672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c5eb6-7249-4086-92bf-75c7b32f0e9d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:43.258660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:43.259312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:43.259591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:43.259861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] PATH_INFO: `/v3/users/651f54adbe2148c094738bcf934e8a8c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:43.260981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] RBAC: Authorizing `identity:get_user(user_id=651f54adbe2148c094738bcf934e8a8c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:43.262411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:43.264002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 593/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Sep 1 16:16:43 2022] GET /identity/v3/users/651f54adbe2148c094738bcf934e8a8c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:43.270763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:43.367596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1e7d-90d4-474d-aadf-5b052cd326c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:43.377265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:43.399993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:43.400578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:43.401020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:43.401130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:43.466836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:43.468303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:43.563123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 605/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:16:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:16:43.569201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:43.596347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:43.667702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff161779-f869-442c-b008-d5df36fe3eba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:43.700858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:43.701474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:43.701755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:43.702026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] PATH_INFO: `/v3/projects/64997bc8748642b0bd578e4146842613` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:43.703132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] RBAC: Authorizing `identity:get_project(project_id=64997bc8748642b0bd578e4146842613)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:43.704587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd27ef-ea83-466d-b7c7-d608a6378824 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:43.706193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 594/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Sep 1 16:16:43 2022] GET /identity/v3/projects/64997bc8748642b0bd578e4146842613 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:43.712329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:16:43.724151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:43.810397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a0ce9-1861-4e7f-b93c-b4001c60b7ee None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:16:43.842875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:16:43.843490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:16:43.843767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:16:43.844035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:16:43.844753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=64997bc8748642b0bd578e4146842613, user.id=651f54adbe2148c094738bcf934e8a8c) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:16:43.845056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:16:43.846463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a44813-1d42-4f3a-9173-03c9fb1b15fd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:16:43.852695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 606/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:16:43 2022] GET /identity/v3/role_assignments?user.id=651f54adbe2148c094738bcf934e8a8c&scope.project.id=64997bc8748642b0bd578e4146842613&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:16:53.696782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=53 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:16:53.850103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=63 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:57.973496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bc84-2fb4-4051-82ca-2cf11564dd52 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.974097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bc84-2fb4-4051-82ca-2cf11564dd52 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.974097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148bc84-2fb4-4051-82ca-2cf11564dd52 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:57.975434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 595/1201] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:17:57.980960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:57.983514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:57.983514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.000434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:17:58.001625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:17:58.069658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 607/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:17:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:17:58.338771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bc84-2fb4-4051-82ca-2cf11564dd52 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.439790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148bc84-2fb4-4051-82ca-2cf11564dd52 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.473883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.474509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.474788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.475059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.485019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.486508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:17:58.502495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:17:58.506399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:17:58.507705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:17:58.507705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:17:58.507705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:17:58.507705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 596/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:17:58.508750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.606887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa9e1f7-f680-4e0e-9edf-3c636af4c6df None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.635470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.642015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.642614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.642894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.643234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.643876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:17:58.644174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.645531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bacf8a-986f-4da4-8cb3-811ad6344b51 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.656687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 608/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:17:58.662635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:17:58.715475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:17:58.761325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc279a70-215a-402e-9fef-a636d9e16615 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:17:58.794796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:17:58.795484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:17:58.795761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:17:58.796032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:17:58.796677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:17:58.798157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31dea2-a377-4454-acff-4811be95e5ae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:17:58.872817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 597/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Sep 1 16:17:58 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:00.873957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c76fa-dfb0-4a9d-a4c8-0885f3b738d5 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:00.874508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c76fa-dfb0-4a9d-a4c8-0885f3b738d5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:00.874633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c76fa-dfb0-4a9d-a4c8-0885f3b738d5 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:00.876006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 609/1206] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:00.881251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:00.881690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:00.881791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:00.900546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:00.901707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:00.968149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 598/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:01.243594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c76fa-dfb0-4a9d-a4c8-0885f3b738d5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:01.346956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c76fa-dfb0-4a9d-a4c8-0885f3b738d5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:01.385168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:01.385799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:01.386073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:01.386339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:01.397113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:01.402136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:01.403106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:01.404230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 610/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:01.409814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:01.451245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:01.512984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bcd26-f692-4e41-8645-3c01d9a44607 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:01.550020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:01.550644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:01.556395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:01.556395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:01.556395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:01.556395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:01.556395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:01.562769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 599/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:01.568617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:01.623177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:01.673089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea42abe-cb2b-4ca1-b19a-60e455dc118d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:01.706055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:01.706669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:01.706944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:01.707240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:01.717448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:01.719019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:01.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:01.739858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:01.741107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:01.741107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:01.741107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:01.741107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 611/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:01.741389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:01.839470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:01.842827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6b8a4-ecde-48be-b589-3395b856552c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:01.875853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:01.876453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:01.876726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:01.876991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:01.877657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:01.877952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:01.879342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:01.890062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 600/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:01.896187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.000843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc715dc-22a6-4b65-b551-88f88fcac4c9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.002402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.035574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:02.036899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.036899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.036899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.043521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:02.045486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:02.057656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:02.058779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:02.059973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 612/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:02.065354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.167241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe87e4-8182-44b9-8bf9-9c9f866b522f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.201504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:02.202001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.202281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.202556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.203316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:02.203625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:02.205059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:02.221842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 601/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:02.227800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.327889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328833af-faa8-481a-833a-94d7790e9c3d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.361481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:02.362081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.362358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.362630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.392427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.430337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:02.431769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:02.527685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 613/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:02.534158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.553885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.638810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d356e-2378-45b9-be53-da948bb94879 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.674385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:02.675013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.675339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.675607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.687236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:02.688690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:02.705278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:02.706412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:02.707730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:02.707730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:02.707730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:02.707730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 602/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:02.712096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.773491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.812713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a102e-16f4-45db-a1ea-3fe54981972b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:02.847438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:02.848038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:02.848349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:02.848642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:02.849281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:02.849583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:02.850936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:02.862727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 614/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:02.868529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:02.937422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:02.968291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5f56-caff-4a6a-b638-625927781078 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.001351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.001970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.003640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.003640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.009376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.010825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:03.022906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:03.024379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:03.025395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 603/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:03.030913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.131852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa162b-f0c9-4dda-ac2b-e503e0d1cfc4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.153731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.166037 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.166698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.166974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.167272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.168033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:03.168334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.170855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb1915-f37f-411b-b066-890ab92ea0aa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.182151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 615/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:03.188657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:03.290861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4d183-9068-416f-a67a-b6b655d59113 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:03.319379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:03.325057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:03.325658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:03.325959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:03.326412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:03.327383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=0688b8d8c0aa46818633fbc17dfb58d7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:03.327823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:03.329662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc9a4c6-b9d5-44d1-b333-16753de651d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:03.334996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 604/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:03 2022] GET /identity/v3/role_assignments?user.id=0688b8d8c0aa46818633fbc17dfb58d7&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:05.277789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0510d-d59c-4736-b252-151f2337b818 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.281974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0510d-d59c-4736-b252-151f2337b818 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.281974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0510d-d59c-4736-b252-151f2337b818 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.281974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 616/1220] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:05.285495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.285929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.286063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.305574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:05.322366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:05.379346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 605/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:05.652468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0510d-d59c-4736-b252-151f2337b818 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:05.754911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0510d-d59c-4736-b252-151f2337b818 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:05.788090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:05.788707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.788983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.789249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.799437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:05.804442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:05.805752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:05.806986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 617/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:05.811848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:05.908283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1d370-94dc-4e9a-83c9-51d752aa91f3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:05.931285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:05.941576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:05.942180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:05.942457 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:05.942727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:05.943413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:05.943709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:05.945077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:05.953350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 606/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:05.959062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.002557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.057921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c30f1-9112-4e8e-be50-1c80f7109941 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.092228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.092838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.093112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.093379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.103854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.105272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:06.119251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:06.120378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:06.121301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:06.121301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:06.121301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:06.121301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 618/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:06.126884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.225615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d7b32-5fdf-4170-8296-a6c932b14616 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.259243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.259862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.260143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.260417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.261061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:06.261362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.262728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.274975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 607/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:06.281117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.374455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.381798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e2551-9ae1-4823-8b24-3bf791156aca None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.417414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.418046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.418326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.418598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.426157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.432169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:06.439872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:06.440991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:06.442002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 619/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:06.446935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.493753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.544321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e741a-c03d-4a97-b9d9-d08d646fffb3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.577606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.578207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.578484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.578756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.579499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:06.579857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.581287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.598408 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 608/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:06.604385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.706518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd337c8-e546-40e7-8133-68a3514ee5ed None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.741218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.741848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.742129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.742395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.753896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.755432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:06.769958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:06.771127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:06.772282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:06.772282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:06.772282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:06.772282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 620/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:06.776970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.877478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d992f4-c507-4d4f-9b27-1e5b2df11e0e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:06.879842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:06.910961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:06.911600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:06.911875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:06.912147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:06.912779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:06.913077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:06.914441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:06.925914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 609/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:06.931969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:06.942303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.031988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc14da3-d03d-49dd-8e68-25f878bc020a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.065464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.066086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.066362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.066631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.073346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.074775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:07.086009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:07.087073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:07.088281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 621/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:07.093481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.193293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a19283-34ee-4b2c-b06c-465ce169b779 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.227707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.228299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.228572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.228837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.229516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:07.229813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.231281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17bedc3-dff5-4f3f-9c46-270726131a57 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.243641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 610/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:07.249878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:07.317132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:07.350721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087707c4-fd92-430b-b1bd-26e338e8ddc2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:07.384224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:07.384877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:07.385249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:07.385520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/0688b8d8c0aa46818633fbc17dfb58d7/roles/bf996b24cd4844aa84d4e9aa5ccd54da` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:07.399936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=0688b8d8c0aa46818633fbc17dfb58d7, role_id=bf996b24cd4844aa84d4e9aa5ccd54da)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:07.401361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2156add1-90a9-43bf-b4cb-e7268efcf875 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:07.530902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 622/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:07 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/0688b8d8c0aa46818633fbc17dfb58d7/roles/bf996b24cd4844aa84d4e9aa5ccd54da => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:09.444191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4749-7fba-44fc-b036-6b31303bf219 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.444712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4749-7fba-44fc-b036-6b31303bf219 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.444712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff4749-7fba-44fc-b036-6b31303bf219 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.446054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 611/1233] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:09.451300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.451701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.451842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.469580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:09.471942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:09.536627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 623/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:09.811689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff4749-7fba-44fc-b036-6b31303bf219 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:09.836134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:09.910075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff4749-7fba-44fc-b036-6b31303bf219 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:09.943430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:09.944050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:09.944328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:09.944599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:09.954723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:09.959603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:09.965054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:09.966670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 612/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:09.967554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.013738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.067102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9cf5a-8774-4db6-b326-abb7c885205b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.099191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.099794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.100073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.100344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.101005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.101308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.102670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.110998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 624/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Sep 1 16:18:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.116819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.201354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.217255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbce373-9b9c-47df-baa3-5e18a02a0a13 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.249881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.250507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.250785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.251055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.261015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.262461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:10.277326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:10.278423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:10.280062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:10.280062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:10.280062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:10.280062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 613/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:10.285029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.383829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5b216-d4c6-48d3-aeb1-9d7899893993 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.393206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.417791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.418389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.418663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.418931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.419721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.419946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.421305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.433563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 625/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.439380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.539605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b843b04-a2ed-40ea-abff-fdd29f463158 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.573197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.573822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.574096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.574363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.577692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.582187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.583653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:10.594887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:10.596006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:10.597147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:10.599292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 614/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:10.601788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.701095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab63ac-49cf-4486-ac61-5fc1c079c8db None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.737173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.737785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.738042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.738307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.739024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:10.739348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.740752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:10.758438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 626/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:10.765524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:10.864802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13cf12f-7496-4e65-9cc3-652cc079ce1a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:10.898921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:10.899574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:10.899849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:10.900121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:10.960466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:10.966915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:10.968439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.067662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 615/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:11.073747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.100734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.171970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4d8d7-85e5-459b-b0e8-1a826ed8e493 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.206931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.207571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.207846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.208114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.218663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.220194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:11.235496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:11.236615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:11.237739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:11.237739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:11.237739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:11.237739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 627/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:11.242823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.336728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.343374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679caf2-89fb-4ae3-94e5-c3d6cef858ae None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.377443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.378167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.378441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.378711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.379620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.379919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.381372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.392636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 616/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:11.398428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.479044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:11.501170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7578c-fc85-4084-8cb3-3cfc2c5e4801 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.535294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.535906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.536180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.536446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.542831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.544426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:11.556349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:11.563419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:11.564508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 628/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:11.565111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.664016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109300-45b6-445e-86b2-c163e5d39bb6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.697185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.697777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.698050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.698313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.699006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.699319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.704548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b48705-6286-48cf-b6bc-2aedbf9ad6be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.712671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 617/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:11.718825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:11.817654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b9986-84b1-4dd7-a88e-3b315f3dfaa1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:11.850034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:11.850628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:11.850905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:11.851197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:11.851918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] RBAC: Adding query filter params (role.id=bf996b24cd4844aa84d4e9aa5ccd54da, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=0688b8d8c0aa46818633fbc17dfb58d7) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:11.852318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:11.853738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aeeb0c-1b3f-422f-8d2d-84402bdcab55 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:11.860009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 629/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:11 2022] GET /identity/v3/role_assignments?user.id=0688b8d8c0aa46818633fbc17dfb58d7&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=bf996b24cd4844aa84d4e9aa5ccd54da => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:13.835852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320b67-6d54-41d7-8e09-4a82b01c655a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.836465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320b67-6d54-41d7-8e09-4a82b01c655a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.836465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40320b67-6d54-41d7-8e09-4a82b01c655a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.837799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 618/1247] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:13.843057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:13.843489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:13.843626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:13.849877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:13.862518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:13.863743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:13.929180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 630/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:14.198346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40320b67-6d54-41d7-8e09-4a82b01c655a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.224918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.312238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40320b67-6d54-41d7-8e09-4a82b01c655a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.345502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.346163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.346485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.346756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.357470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:14.362476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.363553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:14.364966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 619/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:14.369666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.469097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ad81f-c05e-4ca7-b7cf-42a530f329f7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.498736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.503569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.504166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.504439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.504703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.505358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.505653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.506996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.515422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 631/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.521190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.597787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.619685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2b184-18de-45ad-86e9-f38eba236d3b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.653053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.653700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.654105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.654377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.664555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.666038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:14.681044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:14.682182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:14.683355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:14.683355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:14.683355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:14.683355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 620/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:14.688905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.789781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ee131-5fbe-4dd8-92d5-cf1565c7cd2a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.827454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.828052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.828325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.828588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.829225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:14.829520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:14.830891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:14.842589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 632/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:14.848257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:14.956637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f1931-fd14-4fb9-9622-2d764d94acd1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:14.982462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:14.990226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:14.990906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:14.991206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:14.991484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:14.999258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.000698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:15.011605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.014778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:15.016010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 621/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.019544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.074077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.116575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5fcbf-b46c-451e-9311-a325ec6b606c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.148695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.151283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.151283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.151283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.151283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.151283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.152257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.169913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 633/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.175841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.275399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da49960-ae06-491c-9aae-0228e276d599 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.309630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.310236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.310515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.310787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.365088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.376971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.378420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.473936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 622/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:15.479927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.578751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aafd09f-24d0-4f5f-b862-dc5eb08baf97 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.592252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.612589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.613208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.613483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.613752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.624701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.624952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:15.640305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.641459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:15.642561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:15.642561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:15.642561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:15.642561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 634/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.647288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.733729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:15.745020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dfc453-0906-4bc2-abd9-3b017d53a792 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.777766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.778920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.779220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.779490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.780120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:15.780418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.781779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.792534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 623/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:15.798616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:15.898846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036728e5-ca8f-4a56-9c2f-6047d5986e75 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:15.930950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:15.931593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:15.931865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:15.932129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:15.939202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:15.940617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:15.952234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:15.955875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:15.956950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 635/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:15.959280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.056992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679d28f-fda6-46d4-b259-62f149106792 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.090921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.091811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.092094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.092363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.093044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.093344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.094918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e93fd5-16b8-4f5e-866d-0e270789a252 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.106715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 624/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:16.112947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:16.139210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:16.211478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b289ca2-c82e-47cf-a51c-558ceeaf09f4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:16.244694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:16.245296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:16.245949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:16.245949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:16.246535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=0688b8d8c0aa46818633fbc17dfb58d7) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:16.246837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:16.248391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60027e-21f9-4c32-8823-cff8ea21fa89 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:16.253796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 636/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:16 2022] GET /identity/v3/role_assignments?user.id=0688b8d8c0aa46818633fbc17dfb58d7&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:18.155743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b877e-269c-422d-a042-8e69b338f950 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.156276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b877e-269c-422d-a042-8e69b338f950 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.156276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b877e-269c-422d-a042-8e69b338f950 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.157689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 625/1261] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:18.162742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.163123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.163336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.182034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:18.183282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:18.251291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 637/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:18.551600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b877e-269c-422d-a042-8e69b338f950 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.563218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.651376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b877e-269c-422d-a042-8e69b338f950 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.685421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.686038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.686310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.686658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.696758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:18.701902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:18.703026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:18.703955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 626/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:18.709073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.727225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.806745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f4e9a8-0112-44c5-b076-ec4b4732a5e8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:18.847492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:18.848119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:18.848398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:18.848671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:18.849345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:18.849646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:18.851020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:18.859828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 638/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:18.865763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:18.935274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:18.969731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722094b-944f-4bb9-8089-205b4d6dfa5c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.002160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.002782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.003059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.003356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.013625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.015050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:19.029115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.030246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:19.031606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:19.031606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.031606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:19.031606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 627/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:19.036022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.118137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.136620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85c841-a068-41b6-8a2e-e3c1d22566da None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.169765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.170374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.170630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.170898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.171604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.171907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.173272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.185232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 639/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.190967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.291534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15256c-5332-4958-936a-554f0d9c8815 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.312734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.326157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.326786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.327067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.327364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.334861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.336332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:19.347351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.348481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:19.349658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 628/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:19.354341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.459430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480935e8-6e99-4dc3-b5d9-dd3ceb089d0f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.493022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.493734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.496031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.496031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.496031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.496031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.496704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.506386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.515254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 640/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.521023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.620980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde409be-5920-4bfe-86c8-c68ede447b5f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.653995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.654608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.654880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.655144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.665382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.666820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:19.681924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:19.682862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:19.684498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:19.684498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:19.684498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:19.684498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 629/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:19.689038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.787949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1ac4-a281-43d3-a0d6-fd862c98afb2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.822522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.823121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.823426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.823697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.824335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:19.824638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.826105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.837123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 641/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:19.843904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:19.875736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:19.943107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19725ea8-8eda-4df7-9cca-84791c6dc59e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:19.977481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:19.978095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:19.978369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:19.978637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:19.985674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:19.987105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:19.998558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:19.999650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:20.000802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 630/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:20.005538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.063983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.105291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97139f-66b3-459b-854d-535f5b551f0a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.139301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.139935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.140321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.140592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.141327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:20.141627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.143035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb10021-17d6-4e6e-bc8b-174cf2330923 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.156720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 642/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:20.162859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:20.255345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:20.265897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9bed5-8d84-4303-9bea-942d9ac79a02 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:20.298052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:20.298769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:20.299051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:20.299349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] PATH_INFO: `/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/0688b8d8c0aa46818633fbc17dfb58d7/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:20.301494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7b628bcf00bc4a80be6a5ef7a15084c3, user_id=0688b8d8c0aa46818633fbc17dfb58d7, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:20.302942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208743a-38dc-4c3c-8cd4-94e30662a0b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:20.484818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 631/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Sep 1 16:18:20 2022] PUT /identity/v3/projects/7b628bcf00bc4a80be6a5ef7a15084c3/users/0688b8d8c0aa46818633fbc17dfb58d7/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:18:22.391309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3122-48e1-4af9-a3d9-514f69129649 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.391828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3122-48e1-4af9-a3d9-514f69129649 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.391919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe3122-48e1-4af9-a3d9-514f69129649 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.393185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 643/1274] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:22.398364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.398782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.398901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.417207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:22.418393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:22.486001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 632/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:22.770416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe3122-48e1-4af9-a3d9-514f69129649 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:22.874428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe3122-48e1-4af9-a3d9-514f69129649 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:22.891648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:22.907236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:22.907850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:22.908125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:22.908392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:22.918033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 01 16:18:22.923610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:22.924671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:22.925526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 644/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:22.931225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.019246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.037384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e66f0-5e3a-4ca9-93d7-4003840aadec None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.071992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.072588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.072888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.073152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.073802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.074093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.075990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.085298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 633/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Sep 1 16:18:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.091399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.193113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700256c5-6d1b-4b93-b394-c5a41d3f9290 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.225952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.226567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.226838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.227107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.237860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.239409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:23.258560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:23.259569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:23.263683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:23.263683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:23.263683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:23.263683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 645/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 01 16:18:23.265088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.368835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04ab2-67e2-421e-b389-bf60b73a4641 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.400899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.401506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.401761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.402030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.402666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.403497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.404860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.416320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 634/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.422607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.453311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.520947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e1fbe-3128-417c-be8e-c7e5d84e4033 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.555028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.555829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.556104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.556376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.563413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.565440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 01 16:18:23.576806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:23.577896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:23.578983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 646/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:23.583974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.587814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.681595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8ae72-d465-4614-8ccd-a46ea534bbd0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.715227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.716569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.716850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.717121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.717846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:23.718148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.719606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:23.741580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 635/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:23.747670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:23.843319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:23.849459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ebf16-046a-4423-968c-077e03e34b57 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:23.882504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:23.883099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:23.883401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:23.883671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:23.951216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:23.952665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.040174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.050685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 647/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Sep 1 16:18:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 01 16:18:24.056851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.117938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.156853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68fcc4-74c2-4bbf-8b24-d3c42d39f5d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.189906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.190574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.190852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.191121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.202158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.203591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 01 16:18:24.217964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.219478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 01 16:18:24.220615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 01 16:18:24.220615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 01 16:18:24.220615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:24.220615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 636/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.225009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.329828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd2345-7b0e-4eaf-9878-1fadfc6b0dea None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.365592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.366176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.366452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.366722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.367391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:24.367687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.369043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.380680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 648/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:24.386391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.486422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8fe37-1e85-4898-b26d-8f87b0df21bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.500343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.519570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.520196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.520479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.520750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.527275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.528920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 01 16:18:24.539914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:24.540982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Sep 01 16:18:24.541943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 637/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:24.547623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.598047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:24.646804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd943e1-eb0b-4163-83a9-5d9dd87d4f93 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.679886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.680488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.680765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.681037 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.681757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:24.682062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.683516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c02a99-bc5b-4b40-bdd7-e7d0067a8859 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.695817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 649/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:24.702152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:24.801066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b2705-ad5b-4333-8e68-2492f7cca4d1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:24.834051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:24.834643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:24.834918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:24.835209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:24.835948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] RBAC: Adding query filter params (role.id=9730e643e75447a485d531fe8f00c6c5, scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3, user.id=0688b8d8c0aa46818633fbc17dfb58d7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:24.836231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:24.841416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91c7a1-f70e-4791-b1a2-d0efa00c98f8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:24.847071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 638/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Sep 1 16:18:24 2022] GET /identity/v3/role_assignments?user.id=0688b8d8c0aa46818633fbc17dfb58d7&scope.project.id=7b628bcf00bc4a80be6a5ef7a15084c3&role.id=9730e643e75447a485d531fe8f00c6c5 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:26.829675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544622-e705-4395-98ac-261c1239456a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.830206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544622-e705-4395-98ac-261c1239456a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.830206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd544622-e705-4395-98ac-261c1239456a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.831622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 650/1288] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:26.836589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:26.837010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:26.837143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:26.855017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:26.856858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:26.886134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:26.922289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 639/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:27.201109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd544622-e705-4395-98ac-261c1239456a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.305663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd544622-e705-4395-98ac-261c1239456a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.316467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:27.339002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.339649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.339921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.340820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.341407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.342825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:27.351425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:27.352663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:27.353515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 651/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:27.359492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.456852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f84368-cc98-4de8-ac42-7bb49dd80c02 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.490187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.492354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.492354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.492354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.492354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.492354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.493634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c38522-7dcd-474a-8205-cc25987aed5f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.498559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 640/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:27.504430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:27.603089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a59baaf-2ccd-423a-8c36-ee38569eac72 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:27.636063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:27.636657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:27.636932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:27.638004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:27.638632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:27.638930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:27.640315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701704a8-03a5-43ac-9f30-a14ef2cd9a75 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:27.644649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 652/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 16:18:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:29.569974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f24fe-08de-4908-bd7a-288f6c55760e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:29.570528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f24fe-08de-4908-bd7a-288f6c55760e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:29.570528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f24fe-08de-4908-bd7a-288f6c55760e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:29.571876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 641/1293] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:29.577818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ac8a-ddec-4879-817b-c8151aad8cd9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:29.578237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ac8a-ddec-4879-817b-c8151aad8cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:29.578364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760ac8a-ddec-4879-817b-c8151aad8cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:29.596535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0760ac8a-ddec-4879-817b-c8151aad8cd9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:29.597715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0760ac8a-ddec-4879-817b-c8151aad8cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:29.626706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:29.663574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 653/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:29.928331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f24fe-08de-4908-bd7a-288f6c55760e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:29.959321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:30.026558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f24fe-08de-4908-bd7a-288f6c55760e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:30.061151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:30.061745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:30.062021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:30.062292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:30.062935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:30.064331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da18d7-ee8d-4520-93c6-b65a65299457 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:30.170150 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 642/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Sep 1 16:18:29 2022] POST /identity/v3/services => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:32.074242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c8e-535e-41ff-9c06-754d91d1a0f8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.075374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c8e-535e-41ff-9c06-754d91d1a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.075601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995c8e-535e-41ff-9c06-754d91d1a0f8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.076948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 654/1296] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:32.082111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.082535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.082646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.101013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:32.102195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:32.173761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 643/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:32.441560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995c8e-535e-41ff-9c06-754d91d1a0f8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.541268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995c8e-535e-41ff-9c06-754d91d1a0f8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.580699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.581337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.581609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.581877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.582459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.583841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:32.592878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:32.595232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:32.596232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 655/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:32.600042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.624363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.707630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8a23-6356-4be1-a115-32d89e9dbffe None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.740417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.741016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.741289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.741555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.742172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:32.742468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.743858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.750066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 644/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:18:32.756209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.787537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:32.856015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14acdb-2721-4ffc-b064-c8630e045546 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:32.890056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:32.890651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:32.890924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:32.891220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:32.891958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] RBAC: Adding query filter params (interface=public, service_id=e7e17850159a4831a4a2943606016c14, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:32.892258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:32.893604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c873a770-7a3c-4112-95b8-a63318dafbfc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:32.898565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 656/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/endpoints?service_id=e7e17850159a4831a4a2943606016c14&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:32.905831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:32.988901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:33.006818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae25699e-ca0b-4918-84f8-d68239dcdb07 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:33.038779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:33.039406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:33.039688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:33.039962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:33.040558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:33.041969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d75748-b29b-4efe-a82a-60411a2c713d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:33.049204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 645/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Sep 1 16:18:32 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:18:34.975728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0696-5ddd-464b-81c8-b11030ce9980 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.976385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0696-5ddd-464b-81c8-b11030ce9980 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.976385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d0696-5ddd-464b-81c8-b11030ce9980 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:34.977647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 657/1302] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:18:34.982828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:34.983298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:34.983399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.001344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:18:35.002515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:18:35.068600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 646/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:18:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 01 16:18:35.336509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d0696-5ddd-464b-81c8-b11030ce9980 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.430996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.438141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d0696-5ddd-464b-81c8-b11030ce9980 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.471250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.471871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.472149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.472421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.473000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.474408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 01 16:18:35.483094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 01 16:18:35.484494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Sep 01 16:18:35.485630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 658/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 01 16:18:35.490182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.565220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:35.587463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb136ea-76a6-4f86-ab8b-10369d4c77f0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.624940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.625548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.625887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.626154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.626813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:18:35.627184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.631347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ef9f2-f19f-48d0-b29d-ca5cf3271307 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.633829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 647/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Sep 1 16:18:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:18:35.640583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:18:35.740851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dd5f0-51e5-4f12-bf51-b81bff3ee5e5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:18:35.773685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:18:35.774275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:18:35.774553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:18:35.774818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:18:35.775515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:18:35.776870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e39f4-33ff-41bd-8bf7-52397be281d4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:18:35.878131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 659/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:18:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:18:45.671912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:18:45.873464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:32.292148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab7d38-cbb5-4fd7-9301-81111808797b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.292744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab7d38-cbb5-4fd7-9301-81111808797b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.292744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab7d38-cbb5-4fd7-9301-81111808797b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.299396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b614c-5456-4a05-a1da-fbeb4c466fdb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.299876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b614c-5456-4a05-a1da-fbeb4c466fdb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.299980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b614c-5456-4a05-a1da-fbeb4c466fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.304895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 648/1307] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:32.395764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-476b614c-5456-4a05-a1da-fbeb4c466fdb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:32.396981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-476b614c-5456-4a05-a1da-fbeb4c466fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:32.593146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 660/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:24:32.913980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.914560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.915022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:32.916729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 649/1309] 10.222.0.16 () {54 vars in 965 bytes} [Thu Sep 1 16:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:32.927913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86be5-439a-4daf-a39c-c6bf4c912c8f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:32.928216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86be5-439a-4daf-a39c-c6bf4c912c8f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:32.928469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb86be5-439a-4daf-a39c-c6bf4c912c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.050371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7bb86be5-439a-4daf-a39c-c6bf4c912c8f None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:33.051836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7bb86be5-439a-4daf-a39c-c6bf4c912c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.082830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:33.175962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 661/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Sep 1 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:33.186215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:33.188212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.316769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b16d6a-4c98-4e05-a413-369c06196a39 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:33.405193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:33.405816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:33.406169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:33.406575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:33.408291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:33.472097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:33.534162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:33.535802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddbf5f-5f08-44b5-8d92-20520f147938 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:33.556518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 650/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:35.795698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.796239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.796239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.797611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 662/1312] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:35.802771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05825-49d3-4a6d-bd56-8566f698849c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:35.803230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05825-49d3-4a6d-bd56-8566f698849c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:35.803345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e05825-49d3-4a6d-bd56-8566f698849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:35.882968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-30e05825-49d3-4a6d-bd56-8566f698849c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:35.884866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30e05825-49d3-4a6d-bd56-8566f698849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:35.987769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 651/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:36.271637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:36.273165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:36.378412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc744dee-6ec9-49c2-b8a3-e8db72bfbd9f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:36.450074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:36.452551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:36.453152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:36.453432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:36.453699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:36.456117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:36.563213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:36.564643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48135e6-071b-4234-a889-4b39cc2cd99e service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:36.583258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 663/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:39.047281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.047848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.047848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.049168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 652/1315] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:39.054341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794bba-68e7-4795-9048-e5a5e4eef3b2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.054743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794bba-68e7-4795-9048-e5a5e4eef3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.054884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91794bba-68e7-4795-9048-e5a5e4eef3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.121186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-91794bba-68e7-4795-9048-e5a5e4eef3b2 None None] MFA Rules not processed for user `3cbff81e600346ad949641c0f50e4198`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:39.122406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91794bba-68e7-4795-9048-e5a5e4eef3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.172585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.216612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 664/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:39.501865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:39.503438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.513388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.606677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc2755-db85-4fd1-9e02-78603f2d5189 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:39.678001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:39.678602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:39.678880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:39.679175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:39.680764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:39.742982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:39.789136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:39.790626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070032d-8cf9-4eac-ae63-85a28b01f5e4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:39.808682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 653/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:39 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:42.526495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d975b9-1886-42a2-9b68-4fd96878b10f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.527041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d975b9-1886-42a2-9b68-4fd96878b10f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.527041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d975b9-1886-42a2-9b68-4fd96878b10f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.528456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 665/1318] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:42.533473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30c8-10be-4ac5-8397-644b42193632 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:42.533917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30c8-10be-4ac5-8397-644b42193632 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:42.534009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a30c8-10be-4ac5-8397-644b42193632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:42.601660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8f6a30c8-10be-4ac5-8397-644b42193632 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:42.602855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f6a30c8-10be-4ac5-8397-644b42193632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:42.698067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 654/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:43.006252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.006533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.006805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.008178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 666/1320] 10.222.0.16 () {54 vars in 965 bytes} [Thu Sep 1 16:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:43.014310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347751-e601-4836-bc00-d6c9d68141c1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.014594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347751-e601-4836-bc00-d6c9d68141c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.014866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb347751-e601-4836-bc00-d6c9d68141c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.037486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:43.091975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fb347751-e601-4836-bc00-d6c9d68141c1 None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:43.093182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb347751-e601-4836-bc00-d6c9d68141c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:43.168752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 655/1321] 10.222.0.16 () {58 vars in 1062 bytes} [Thu Sep 1 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:43.177744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:43.179333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:43.281656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd87407-690b-4989-bcb0-d9096603d65d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:43.354516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:43.355112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:43.355412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:43.355678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:43.357271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:43.364662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:43.466346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:43.467790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcee6291-0d99-48f1-b3fe-d993d070509a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:43.486172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 667/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:46.890463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.891069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.891069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.892388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 656/1323] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:46.897631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265238b6-1339-4951-8725-b0964797e772 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:46.898039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265238b6-1339-4951-8725-b0964797e772 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:46.898187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265238b6-1339-4951-8725-b0964797e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:46.976671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-265238b6-1339-4951-8725-b0964797e772 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:46.977882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-265238b6-1339-4951-8725-b0964797e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.038692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.079112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 668/1324] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:47.370633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:47.372205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.482060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e2a81-fdaf-43b7-9269-b1eac23414d7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:47.494725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:47.557454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:47.558143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:47.558416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:47.558684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:47.560317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:47.669581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:47.670917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc239fd-685c-478b-831f-ab190ae0a12b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:47.689328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 657/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:49.675055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.675659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.675659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.677013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 669/1326] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:49.682941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092fa9-2e39-48e8-a97a-72565ec2f502 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:49.683383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092fa9-2e39-48e8-a97a-72565ec2f502 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:49.683540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83092fa9-2e39-48e8-a97a-72565ec2f502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:49.731545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:49.751356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83092fa9-2e39-48e8-a97a-72565ec2f502 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:49.752593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83092fa9-2e39-48e8-a97a-72565ec2f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:49.848682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 658/1327] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:50.129027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:50.130604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:50.240214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814e8e9-c34d-4c7e-988b-7c56144c3522 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:50.316651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:50.317183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:50.317460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:50.317786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:50.319405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:50.344859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:50.436032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:50.437473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff02dd-6e29-4cc5-98f2-0ea9c045f45f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:50.457506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 670/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:52.469199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:52.469706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:52.469706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:52.471001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 659/1329] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:52.476471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec639-a0ec-48cb-8e66-0975736aea2f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:52.476955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec639-a0ec-48cb-8e66-0975736aea2f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:52.477047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec639-a0ec-48cb-8e66-0975736aea2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:52.549714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-84cec639-a0ec-48cb-8e66-0975736aea2f None None] MFA Rules not processed for user `3cbff81e600346ad949641c0f50e4198`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:52.550949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-84cec639-a0ec-48cb-8e66-0975736aea2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:52.622289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:52.654498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 671/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:52.937683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:52.939254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:53.074131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87666ee-9c86-43a2-a2c4-6e130b7f126c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:53.094769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:53.148210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:53.148818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:53.149089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:53.149352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:53.151703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:53.262192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:53.263695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9006cf-cc77-40df-a4d0-89f9180f40ea service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:53.282625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 660/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:55.799555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:55.800123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:55.800123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:55.801454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 672/1332] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:55.807118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf83159-7525-40fe-9080-04a7ee7b5c57 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:55.807687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf83159-7525-40fe-9080-04a7ee7b5c57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:55.807995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf83159-7525-40fe-9080-04a7ee7b5c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:55.880382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:55.884574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fdf83159-7525-40fe-9080-04a7ee7b5c57 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:55.886691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdf83159-7525-40fe-9080-04a7ee7b5c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:56.006713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 661/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:24:56.293799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:56.295367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:56.408583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d85611-6014-490e-a7cc-fb118cfeaabd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:56.483784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:56.484329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:56.484602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:56.484865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:56.486577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:56.513268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:56.595502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:56.596935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adee114-fdd4-4aab-b9e2-c63d1e484805 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:56.614582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 673/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:24:58.564306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:58.564866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:58.564866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:58.566189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 662/1335] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:24:58.571363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545f77-883e-4c66-89d6-5f91db67d7f6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:58.571786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545f77-883e-4c66-89d6-5f91db67d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:58.571903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545f77-883e-4c66-89d6-5f91db67d7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:58.637559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f1545f77-883e-4c66-89d6-5f91db67d7f6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:24:58.638740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f1545f77-883e-4c66-89d6-5f91db67d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:58.711625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:58.733802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 674/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:24:59.015015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:24:59.016615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:59.122251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128b0b0-8b74-4dac-907f-5b55e32acbd0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:24:59.150142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:24:59.196209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:24:59.196810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:24:59.197082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:24:59.197343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:24:59.200073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:24:59.308720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:24:59.312061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2e13e-8f75-40e7-b585-dad22b5da524 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:24:59.329564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 663/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:01.916018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:01.916550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:01.916550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:01.918810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 675/1338] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:01.923983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f06777-2346-45a3-a7d2-c5056970c9af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:01.924404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f06777-2346-45a3-a7d2-c5056970c9af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:01.924533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f06777-2346-45a3-a7d2-c5056970c9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:01.989759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83f06777-2346-45a3-a7d2-c5056970c9af None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:01.990968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83f06777-2346-45a3-a7d2-c5056970c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:01.999241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.087334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 664/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:02.370704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:02.372276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:02.477135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc218826-72ba-461a-95ad-7697cf430f5a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:02.549820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:02.550418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:02.550694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:02.550963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:02.552710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:02.588184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:02.659969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:02.661402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9763e1a-3ccc-4f98-8b04-2f0c6e3a07d3 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:02.680217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 676/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:06.854638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:06.855191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:06.855191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:06.856769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 665/1341] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:06.861752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba78e88-c1eb-4fb1-a124-684f68932e19 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:06.862161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba78e88-c1eb-4fb1-a124-684f68932e19 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:06.862305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba78e88-c1eb-4fb1-a124-684f68932e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:06.929300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7ba78e88-c1eb-4fb1-a124-684f68932e19 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:06.930528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7ba78e88-c1eb-4fb1-a124-684f68932e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:07.022396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:07.033731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 677/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:07.324268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:07.325692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:07.430546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c329c53-6013-4498-856d-549b2b2c62d3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:07.469661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:07.507820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:07.508355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:07.508634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:07.508907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:07.510549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:07.620365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:07.622434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00add93f-b05d-463d-aea7-a8b86a45c4a7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:07.640877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 666/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:10.715964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:10.716495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:10.716495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:10.717779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 678/1344] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:10.723004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944baba-c2e4-4c7f-b567-6d29af7db71a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:10.723439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944baba-c2e4-4c7f-b567-6d29af7db71a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:10.723599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944baba-c2e4-4c7f-b567-6d29af7db71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:10.790462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c944baba-c2e4-4c7f-b567-6d29af7db71a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:10.791700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c944baba-c2e4-4c7f-b567-6d29af7db71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:10.810425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:10.892372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 667/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:11.176260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:11.177894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:11.297523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec05d5-be14-4b86-93d7-2ceb5a3e1d90 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:11.377240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:11.377845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:11.378123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:11.378384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:11.379999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:11.424244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:11.492289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:11.493730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907ae5e-22e6-41d3-85b1-f732de826168 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:11.513073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 679/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:25:15.818547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:15.819031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:15.819262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:15.820681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 668/1347] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:15.827086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece50e-f1a4-4a87-b425-99621c38762c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:15.827654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece50e-f1a4-4a87-b425-99621c38762c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:15.827654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dece50e-f1a4-4a87-b425-99621c38762c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:15.901213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7dece50e-f1a4-4a87-b425-99621c38762c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:15.903263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7dece50e-f1a4-4a87-b425-99621c38762c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.006491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:16.009129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 680/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:16.299089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:16.300746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.414668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2a0e1-b9f4-4b53-9125-668eaf055df7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:16.465303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:16.495322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:16.495933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:16.496205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:16.496575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:16.498182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:16.611513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:16.611693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da79b5bd-879f-4373-95db-c983a1e015df service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:16.631653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 669/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:21.601446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cfb1c-7306-4944-a117-81f9849fdb46 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:21.601856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cfb1c-7306-4944-a117-81f9849fdb46 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:21.604821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cfb1c-7306-4944-a117-81f9849fdb46 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:21.605764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 681/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:25:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:21.616929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f705ea-d276-40bd-9041-b60fb79ff127 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:21.618195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f705ea-d276-40bd-9041-b60fb79ff127 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:21.618195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f705ea-d276-40bd-9041-b60fb79ff127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:21.739587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-18f705ea-d276-40bd-9041-b60fb79ff127 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:21.739587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18f705ea-d276-40bd-9041-b60fb79ff127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:21.769766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:21.873206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 670/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:26.191954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09c94-2aef-45ff-bc3f-8a8470cd9bd8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.192567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09c94-2aef-45ff-bc3f-8a8470cd9bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.192567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09c94-2aef-45ff-bc3f-8a8470cd9bd8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.193902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 682/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:26.200575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ab2b-c338-400d-b2d8-159c7dedea73 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:26.201030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ab2b-c338-400d-b2d8-159c7dedea73 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:26.201113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0ab2b-c338-400d-b2d8-159c7dedea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:26.287133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-56c0ab2b-c338-400d-b2d8-159c7dedea73 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:26.288590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56c0ab2b-c338-400d-b2d8-159c7dedea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:26.364282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:26.407602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 671/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:25:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:27.520276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45b68-f375-4c1a-ab26-9a63ae1ece3d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:27.520805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45b68-f375-4c1a-ab26-9a63ae1ece3d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:27.520805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45b68-f375-4c1a-ab26-9a63ae1ece3d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:27.522139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 683/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:27.527213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb39a4-fb52-4da6-81e0-43e32d376fe5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:27.527627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb39a4-fb52-4da6-81e0-43e32d376fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:27.527877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb39a4-fb52-4da6-81e0-43e32d376fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:27.606694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1fbb39a4-fb52-4da6-81e0-43e32d376fe5 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:27.608060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1fbb39a4-fb52-4da6-81e0-43e32d376fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:27.721833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 672/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:31.160006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f3a2-aa47-452e-b379-279bd1b7521d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:31.160563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f3a2-aa47-452e-b379-279bd1b7521d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:31.160563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c34f3a2-aa47-452e-b379-279bd1b7521d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:31.161937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 684/1356] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:31.167095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153826-8a88-4962-8e09-5140a431f256 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:31.167533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153826-8a88-4962-8e09-5140a431f256 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:31.167677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153826-8a88-4962-8e09-5140a431f256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:31.212582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:31.248570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-52153826-8a88-4962-8e09-5140a431f256 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:31.249761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-52153826-8a88-4962-8e09-5140a431f256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:31.405134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 673/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:31.714685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:31.714971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:31.715263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:31.716805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 685/1358] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 16:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:31.725639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a1d7-cd7f-4331-887e-21f340c10226 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:31.726125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a1d7-cd7f-4331-887e-21f340c10226 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:31.726544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047a1d7-cd7f-4331-887e-21f340c10226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:31.814529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e047a1d7-cd7f-4331-887e-21f340c10226 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:31.815927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e047a1d7-cd7f-4331-887e-21f340c10226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:31.870412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:31.949908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 674/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 16:25:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:25:31.960402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:31.962245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:32.119946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15ebe6-679d-460b-8262-f2b4d330975c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:32.198743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:32.248246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:32.249100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:32.249502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:32.249884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:32.251659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:32.346819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:32.348637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a112643-6011-4dfd-94e3-15696abb0234 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:32.368548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 686/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:34.535717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:34.536243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:34.536243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:34.537805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 675/1361] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:34.543134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8df9-f134-488e-a4f1-4df685230670 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:34.549308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8df9-f134-488e-a4f1-4df685230670 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:34.549308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8df9-f134-488e-a4f1-4df685230670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:34.613823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-30de8df9-f134-488e-a4f1-4df685230670 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:34.615018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-30de8df9-f134-488e-a4f1-4df685230670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:34.657478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:34.730282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 687/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:35.021681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:35.024491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:35.183887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf8348-14a4-45d8-9e20-95fd4cab111a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:35.194010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:35.317547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:35.318106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:35.318393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:35.318659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:35.320343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:35.428367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:35.430275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72757c5d-8a0c-446b-831f-3279d9ca7132 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:35.452167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 676/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:37.802281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778bb59-039f-41d7-b2d9-30ddd9dcfad4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:37.802845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778bb59-039f-41d7-b2d9-30ddd9dcfad4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:37.802845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778bb59-039f-41d7-b2d9-30ddd9dcfad4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:37.804190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 688/1364] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:37.810703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa7f5-5ebb-4c67-9cc0-3f34ab382624 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:37.811613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa7f5-5ebb-4c67-9cc0-3f34ab382624 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:37.812087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa7f5-5ebb-4c67-9cc0-3f34ab382624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:37.829672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:37.889105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-84dfa7f5-5ebb-4c67-9cc0-3f34ab382624 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:37.890541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-84dfa7f5-5ebb-4c67-9cc0-3f34ab382624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:38.003926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 677/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:38.302984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:38.303289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:38.303564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:38.305044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 689/1366] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 16:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:38.311063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520d38-2885-4863-8d23-33740046914b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:38.313048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520d38-2885-4863-8d23-33740046914b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:38.313048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2520d38-2885-4863-8d23-33740046914b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:38.398286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b2520d38-2885-4863-8d23-33740046914b None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:38.399763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b2520d38-2885-4863-8d23-33740046914b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:38.440130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:38.661736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 678/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:38.671071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:38.672826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:38.828604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8257ed9-1ad2-49c1-95ef-40ac258cefd9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:38.852105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:38.956705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:38.957346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:38.957690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:38.957962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:38.959621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:39.053897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:39.055527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2950a-0203-420a-b191-8f90bc4b7128 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:39.074295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 690/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:39.446133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6dce-8c2d-43ea-ae84-ec65f326c10c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:39.446670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6dce-8c2d-43ea-ae84-ec65f326c10c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:39.446670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6dce-8c2d-43ea-ae84-ec65f326c10c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:39.448262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 679/1369] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:39.454109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c006a-2690-40d0-a175-6328d60930cc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:39.454423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c006a-2690-40d0-a175-6328d60930cc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:39.454662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c006a-2690-40d0-a175-6328d60930cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:39.490777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:39.526658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a97c006a-2690-40d0-a175-6328d60930cc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:39.528753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a97c006a-2690-40d0-a175-6328d60930cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:39.633377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 691/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:40.220296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:40.220296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:40.220296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:40.223185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 680/1371] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 1 16:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:40.226448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd570-4a30-495f-8229-2858199dcb42 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:40.226911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd570-4a30-495f-8229-2858199dcb42 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:40.227265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bd570-4a30-495f-8229-2858199dcb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:40.310748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e44bd570-4a30-495f-8229-2858199dcb42 None None] MFA Rules not processed for user `4a081c08049447949477e73e0891f1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:40.312002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e44bd570-4a30-495f-8229-2858199dcb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:40.373752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:40.401700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 692/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 1 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:40.410553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:40.412336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:40.525242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8317c1b-dc5a-4e21-a472-de53906cfc68 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:40.574075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:40.616274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:40.616893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:40.617161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:40.617429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:40.619017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:40.747943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:40.749388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569bd06-6261-4294-9cf0-061c4ae1859b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:40.770542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 681/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:41.270950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:41.271514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:41.271514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:41.272863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 693/1374] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:41.278188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0971db0-75a4-44df-b4dd-f747b2275ed7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:41.278513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0971db0-75a4-44df-b4dd-f747b2275ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:41.278787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0971db0-75a4-44df-b4dd-f747b2275ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:41.346437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c0971db0-75a4-44df-b4dd-f747b2275ed7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:41.347700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0971db0-75a4-44df-b4dd-f747b2275ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:41.369963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:41.447363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 682/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:41.743068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:41.744635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:41.890679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed957ad-a3a1-4972-9da2-905c77860f34 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:41.941534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:42.008953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:42.009555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:42.009828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:42.010095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:42.012057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:42.102459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:42.104082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bacf59-bb62-4b3b-9944-001aa8726bca service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:42.123316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 694/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:44.172788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:44.173327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:44.173327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:44.174622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 683/1377] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:44.179978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c48ce-9ec8-4a81-974a-d3a801bbda89 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:44.180383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c48ce-9ec8-4a81-974a-d3a801bbda89 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:44.180528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c48ce-9ec8-4a81-974a-d3a801bbda89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:44.247874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3b2c48ce-9ec8-4a81-974a-d3a801bbda89 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:44.249157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3b2c48ce-9ec8-4a81-974a-d3a801bbda89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:44.264355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:44.367199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 695/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:44.691189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:44.692643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:44.858616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77f6b2-51b9-471e-9a96-d4fa5f10c0b5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:44.862782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:44.977575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:44.978200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:44.978665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:44.979012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:44.981241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:45.078040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:45.079777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b41aa-a47f-4c88-829c-df63468d3a94 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:45.100597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 684/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:47.415091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:47.416113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:47.416400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:47.418555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 696/1380] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:47.424140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868d8e-b3ea-46a4-a373-918d485c1e18 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:47.424702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868d8e-b3ea-46a4-a373-918d485c1e18 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:47.424702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868d8e-b3ea-46a4-a373-918d485c1e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:47.436788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:47.500447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a868d8e-b3ea-46a4-a373-918d485c1e18 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:47.501730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a868d8e-b3ea-46a4-a373-918d485c1e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:47.611211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 685/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:47.912897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:47.914610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:48.058720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:48.072594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d5206-e4bc-487f-8b64-9119a80218b2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:48.205772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:48.206467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:48.206879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:48.207299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:48.208995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:48.311746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:48.313373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7ac39-d63c-4bf6-89e7-b6702d8f0475 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:48.323324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:48.331760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 697/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:51.174405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:51.174938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:51.174938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:51.176326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 686/1383] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:51.181721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3bb0-c57e-47a4-aea1-e90c9ad67d82 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:51.182186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3bb0-c57e-47a4-aea1-e90c9ad67d82 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:51.182305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a3bb0-c57e-47a4-aea1-e90c9ad67d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:51.278998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-164a3bb0-c57e-47a4-aea1-e90c9ad67d82 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:51.280935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-164a3bb0-c57e-47a4-aea1-e90c9ad67d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:51.412513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 698/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:25:51.702998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:51.704807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:51.799815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:51.863816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a214771-7096-42b8-a60e-05aa305301e3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:52.012066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:52.012812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:52.013098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:52.013373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:52.015049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:52.078924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:52.127543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:52.129088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b89a1-9a89-45d8-84e7-58fb09f81ff4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:52.147858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 687/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:54.125491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.126291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.126291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.127427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 699/1386] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:54.132763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347ffc-a106-43ba-96ef-229f67cb49fa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.133199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347ffc-a106-43ba-96ef-229f67cb49fa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.133311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347ffc-a106-43ba-96ef-229f67cb49fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.201777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d6347ffc-a106-43ba-96ef-229f67cb49fa None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:54.203102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6347ffc-a106-43ba-96ef-229f67cb49fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:54.308429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 688/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:54.600592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:54.602114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:54.681498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:54.754436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20a448-b871-449a-be7f-47f5c94e608c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:54.869114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:54.869632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:54.869900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:54.870171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:54.871778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:54.910447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:54.973840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:54.976040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3608-f6a8-45dd-b017-66d600e5b9f3 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:55.000289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 700/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:25:56.479931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b5b-5266-4da2-a086-619d711e12c6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:56.480501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b5b-5266-4da2-a086-619d711e12c6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:56.480604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3419b5b-5266-4da2-a086-619d711e12c6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:56.481909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 689/1389] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:56.487852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54081d3-1893-4633-9dfb-4371a02936e7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:56.488676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54081d3-1893-4633-9dfb-4371a02936e7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:56.489174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54081d3-1893-4633-9dfb-4371a02936e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:56.560860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d54081d3-1893-4633-9dfb-4371a02936e7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:56.562058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d54081d3-1893-4633-9dfb-4371a02936e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:56.658369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 701/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:56.965218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0276ad-c44f-4563-8de9-fd607e295894 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:56.965649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0276ad-c44f-4563-8de9-fd607e295894 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:56.965988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0276ad-c44f-4563-8de9-fd607e295894 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:56.967411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 690/1391] 10.222.0.16 () {54 vars in 967 bytes} [Thu Sep 1 16:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:56.972447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:56.972737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:56.973013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:56.990065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:57.059013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] MFA Rules not processed for user `e346b0a022ea4218a4bd10c7e6fe6d91`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:57.060337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:57.080605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d01948-f872-4f46-a3c2-236c8c3da052 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.080889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d01948-f872-4f46-a3c2-236c8c3da052 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.081154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d01948-f872-4f46-a3c2-236c8c3da052 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.082467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 691/1392] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:25:57.088120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.088392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.088656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.127577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:57.138033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 702/1393] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Sep 1 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:57.145756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:57.153878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb4fc1f-c555-451b-8970-99fb72ba37f1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:57.160890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:57.162097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:57.240175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:57.240859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.241207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.241480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.243331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:57.280694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:57.282555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 692/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:25:57.378959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:57.380421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba5c67-f648-48cf-8f11-0940c6ed3a1f service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:57.401067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 703/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:25:57.616963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:57.619578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:57.759714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:57.770151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffdcf63-f0fc-46bc-9214-635c7a8a13cf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:57.899316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:57.900081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:57.900425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:57.900626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:57.902327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.007479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:58.009277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e224113-3272-4915-b2a0-890b4c9fc524 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:58.025390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:58.029107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 693/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:25:58.396027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f67380-cce8-470b-8270-42452e16f52a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:58.396520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f67380-cce8-470b-8270-42452e16f52a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:58.396679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f67380-cce8-470b-8270-42452e16f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:58.480082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-26f67380-cce8-470b-8270-42452e16f52a None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:58.481797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-26f67380-cce8-470b-8270-42452e16f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.572200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:58.578761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 704/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:58.622431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:58.622910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:58.623024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:58.624376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 694/1398] 10.222.0.16 () {54 vars in 943 bytes} [Thu Sep 1 16:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:25:58.630591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2131a0-83a1-4f43-a417-63d4fa4d58c3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:58.630874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2131a0-83a1-4f43-a417-63d4fa4d58c3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:58.631144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2131a0-83a1-4f43-a417-63d4fa4d58c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:58.708436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1c2131a0-83a1-4f43-a417-63d4fa4d58c3 None None] MFA Rules not processed for user `d247a8be3add4804b86dcc0cd4cb72e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:25:58.709634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1c2131a0-83a1-4f43-a417-63d4fa4d58c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.817788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 705/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Thu Sep 1 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:25:58.827369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:25:58.828974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:58.965017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253a22c-d740-48d7-9357-85c552a78c09 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:25:59.053926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:25:59.074813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:25:59.075456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:25:59.075735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:25:59.076002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:25:59.077769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:25:59.163775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:25:59.165988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59797e76-5368-4d29-a05e-022f9d61c622 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:25:59.174473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 695/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:25:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:26:00.097203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:00.097757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:00.097842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:00.099128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 706/1401] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:00.104825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f71b-b74c-4f5c-a9c7-3c7bcdf54c4f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:00.105273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f71b-b74c-4f5c-a9c7-3c7bcdf54c4f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:00.105369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0f71b-b74c-4f5c-a9c7-3c7bcdf54c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:00.178359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5ff0f71b-b74c-4f5c-a9c7-3c7bcdf54c4f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:00.179606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ff0f71b-b74c-4f5c-a9c7-3c7bcdf54c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:00.270160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:00.307234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 696/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:26:00.600154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:00.604291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:00.675301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:00.791966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe722e8-2404-407a-b16e-efd4807f2b37 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:00.934797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:00.935484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:00.935763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:00.936036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:00.937709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:00.960293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:01.052772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:01.054718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46679b95-60c4-4ad9-ac04-302e04ace50b service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:01.077779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 707/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:03.753627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:03.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:03.754238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:03.755611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 697/1404] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:03.761062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb796a-7b89-4541-bce5-a995edb47003 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:03.761560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb796a-7b89-4541-bce5-a995edb47003 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:03.765911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb796a-7b89-4541-bce5-a995edb47003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:03.838907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ebdb796a-7b89-4541-bce5-a995edb47003 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:03.842892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ebdb796a-7b89-4541-bce5-a995edb47003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:03.942936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:03.960873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 708/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:04.250224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:04.251991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:04.419523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ffa73-4cd5-4699-9eae-3576cecd7ce0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:04.490851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:04.557285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:04.557999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:04.558346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:04.558686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:04.560340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:04.660549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:04.662272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fdc56-bd6c-4260-8266-362c340770b8 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:04.681865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 698/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:06.771276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:06.771936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:06.771936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:06.773179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 709/1407] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:06.778542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c46-1eb6-4037-bba6-ac6bef8408ad None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:06.779265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c46-1eb6-4037-bba6-ac6bef8408ad None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:06.779361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce8c46-1eb6-4037-bba6-ac6bef8408ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:06.865512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e3ce8c46-1eb6-4037-bba6-ac6bef8408ad None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:06.866748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3ce8c46-1eb6-4037-bba6-ac6bef8408ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:06.899680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:07.004183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 699/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:07.304345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:07.306908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:07.472501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97012acf-8490-488c-9b2f-006a479ee030 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:07.552091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:07.605543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:07.606195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:07.606472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:07.606749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:07.608507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:07.706647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:07.708280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6a510-26c9-4c24-99a8-6606c1a15c14 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:07.729214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 710/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:17.277141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=37 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:17.726198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=65 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:22.994436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba5f6-87ce-4fea-a302-a04123377021 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:22.995014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba5f6-87ce-4fea-a302-a04123377021 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:22.995014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba5f6-87ce-4fea-a302-a04123377021 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:22.996351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 700/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 1 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:23.001661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34929-d524-4882-a5a8-b3bf13f5405a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:23.002114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34929-d524-4882-a5a8-b3bf13f5405a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:23.002220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34929-d524-4882-a5a8-b3bf13f5405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:23.080375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7fd34929-d524-4882-a5a8-b3bf13f5405a None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:23.081592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7fd34929-d524-4882-a5a8-b3bf13f5405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:23.189287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 711/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 1 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:23.204568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252b3b-720d-4836-a6f7-bb6354273e6a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:23.204847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252b3b-720d-4836-a6f7-bb6354273e6a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:23.205244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a252b3b-720d-4836-a6f7-bb6354273e6a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:23.206476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 701/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Sep 1 16:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:23.211744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8353-3a13-4ca0-ad23-f2713394fea0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:23.212030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8353-3a13-4ca0-ad23-f2713394fea0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:23.212322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8353-3a13-4ca0-ad23-f2713394fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:23.286568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:23.289890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9fea8353-3a13-4ca0-ad23-f2713394fea0 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:23.291093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9fea8353-3a13-4ca0-ad23-f2713394fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:23.406632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 712/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Sep 1 16:26:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:25.779685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:25.780237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:25.780237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:25.781613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:25.787298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45b3a-ef47-47ee-a3cc-32d31c00fab5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:25.787847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45b3a-ef47-47ee-a3cc-32d31c00fab5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:25.788128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c45b3a-ef47-47ee-a3cc-32d31c00fab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:25.855745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-42c45b3a-ef47-47ee-a3cc-32d31c00fab5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:25.856881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-42c45b3a-ef47-47ee-a3cc-32d31c00fab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:25.911976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:25.954832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:26.256268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:26.257853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:26.405732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff7710-79b1-47c1-b1de-75be1429a4dc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:26.479891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:26.519561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:26.520090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:26.520365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:26.520634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:26.522824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:26.612126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:26.613720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47495044-9d56-487c-a994-f0a6ae39e23d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:26.631873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 703/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:27.537263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:27.537818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:27.537818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:27.539137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:27.544101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d407-2312-4380-8376-76d90a8c745a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:27.544493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d407-2312-4380-8376-76d90a8c745a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:27.544679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71d407-2312-4380-8376-76d90a8c745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:27.622662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bb71d407-2312-4380-8376-76d90a8c745a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:27.623906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bb71d407-2312-4380-8376-76d90a8c745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:27.650954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:27.721668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:27.873067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:27.874557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:28.019470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ecd14-bef6-4a41-b714-5be2e6fdf6d9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:28.060016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:28.139192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:28.139712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:28.139989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:28.140253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:28.141852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:28.230798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:28.232571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23145c6c-85b9-4052-a842-22d0540f5f1c service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:28.250651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 715/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:29.158248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.158749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.158879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.160178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:29.165586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b794-45cc-4344-b3a7-881cf0db9380 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.165998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b794-45cc-4344-b3a7-881cf0db9380 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.166146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3b794-45cc-4344-b3a7-881cf0db9380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.219956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:29.243326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-93c3b794-45cc-4344-b3a7-881cf0db9380 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:29.244551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-93c3b794-45cc-4344-b3a7-881cf0db9380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:29.341582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:29.495645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:29.497154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:29.644743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595b2fb-ac51-4cd6-829e-65e0a9dce951 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:29.657421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:29.757941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:29.758552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:29.758825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:29.759094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:29.760716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:29.851624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:29.853244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77f8d-6958-4c7d-8f27-59f0eb60cb84 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:29.871741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 706/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:30.777797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.778379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.778379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.779714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:30.784556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e8c3c-eaa7-4ef9-b457-8812885bca2a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:30.784999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e8c3c-eaa7-4ef9-b457-8812885bca2a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:30.785110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e8c3c-eaa7-4ef9-b457-8812885bca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:30.806423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:30.862356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e78e8c3c-eaa7-4ef9-b457-8812885bca2a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:30.863612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e78e8c3c-eaa7-4ef9-b457-8812885bca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:30.958740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:31.112895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:31.114366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:31.222352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:31.258200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45630ec3-fe03-423b-9132-3e9abb9fcd15 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:31.370096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:31.370619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:31.370893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:31.371175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:31.372780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:31.452754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:31.465980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:31.467662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140792a-3462-4b11-a7e4-353deb4db095 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:31.485896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 718/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:32.344935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.345453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.345453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.346803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:32.352160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf19d6-5936-4fa9-a143-50e527adf7fd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.352617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf19d6-5936-4fa9-a143-50e527adf7fd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.352716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf19d6-5936-4fa9-a143-50e527adf7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.429095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6bdf19d6-5936-4fa9-a143-50e527adf7fd None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:32.430310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6bdf19d6-5936-4fa9-a143-50e527adf7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:32.525731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:32.675721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:32.677799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:32.763777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:32.824933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966fbbc-a659-4a2c-a171-2a0be2627f95 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:32.942552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:32.943209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:32.943489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:32.943763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:32.945695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:32.996133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:33.037450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:33.039095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628197c8-2d77-4c86-b5d4-6f73922044e8 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:33.057468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 709/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:33.919804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:33.920377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:33.920377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:33.921739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:33.926622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072a7d-dea4-4a39-87f1-612ec0794c47 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:33.927052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072a7d-dea4-4a39-87f1-612ec0794c47 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:33.927179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9072a7d-dea4-4a39-87f1-612ec0794c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.002545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a9072a7d-dea4-4a39-87f1-612ec0794c47 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:34.003780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9072a7d-dea4-4a39-87f1-612ec0794c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.100199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:34.260782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:34.262388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.310844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:34.409265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b55ffa-7261-4b79-a5d3-7820de312dcc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:34.526962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:34.527587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:34.527864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:34.528141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:34.529741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:34.537053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:34.618324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:34.619964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7f37c-9b49-4709-8adb-b8c491f3ea7c service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:34.638099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 721/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:35.493001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.493541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.493541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.494893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:35.499839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9014ea-35c0-4251-90d2-80e62c013f13 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:35.500247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9014ea-35c0-4251-90d2-80e62c013f13 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:35.500354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9014ea-35c0-4251-90d2-80e62c013f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:35.581817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ba9014ea-35c0-4251-90d2-80e62c013f13 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:35.583585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ba9014ea-35c0-4251-90d2-80e62c013f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:35.651119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.680392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:35.829772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:35.831380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:35.850452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:35.977858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e78513-94fa-4f61-a7ff-eb76f7bbebbc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:36.063091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:36.088491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:36.089031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:36.089307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:36.089576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:36.091194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:36.179572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:36.181158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c47c2-89e9-46f7-8513-4f9609e81c0e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:36.199105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 712/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:37.094932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:37.095502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:37.095502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:37.096869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:37.102482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb2ce-66f2-45c6-b062-58ecc979ea13 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:37.102927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb2ce-66f2-45c6-b062-58ecc979ea13 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:37.103062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acb2ce-66f2-45c6-b062-58ecc979ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:37.177888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-10acb2ce-66f2-45c6-b062-58ecc979ea13 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:37.179093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-10acb2ce-66f2-45c6-b062-58ecc979ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:37.220289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:37.277133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:37.430033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:37.431523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:37.580466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57de9e-5ae7-4d94-9c25-fa4f47280fd7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:37.634372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:37.693589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:37.694383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:37.694712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:37.694984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:37.696627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:37.785501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:37.787100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aec20f-97fa-43c9-b5b6-32f771c98a8a service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:37.805908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 724/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:38.757726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:38.758318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:38.758318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:38.759751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:38.764792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7501b-82f4-426f-b247-b3cf82645e84 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:38.765226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7501b-82f4-426f-b247-b3cf82645e84 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:38.765334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7501b-82f4-426f-b247-b3cf82645e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:38.957492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6fe7501b-82f4-426f-b247-b3cf82645e84 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:38.958735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6fe7501b-82f4-426f-b247-b3cf82645e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:38.960989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:39.057998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:39.207986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:39.209502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:39.363609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc44e55-e227-468a-91b4-5b6c0e80e692 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:39.390861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:39.476806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:39.477412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:39.477683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:39.477947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:39.479561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:39.567727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:39.569593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d213547-8291-4d40-a21e-c4d2b10c2152 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:39.587435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 715/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:26:41.425963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.426525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.426525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.427868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:41.433050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9c24-5fe9-48bd-8d30-35043ff0754f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:41.433480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9c24-5fe9-48bd-8d30-35043ff0754f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:41.433595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c9c24-5fe9-48bd-8d30-35043ff0754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:41.476003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:41.500720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0b5c9c24-5fe9-48bd-8d30-35043ff0754f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:41.501850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b5c9c24-5fe9-48bd-8d30-35043ff0754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:41.598970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:41.886430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:41.888001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:41.992250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc98a58-0bd2-4cb4-8c95-c87bc6828ccf None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:42.025335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:42.066653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:42.067582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:42.067863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:42.068137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:42.070016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:42.177466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:42.178816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4760e2-324e-4bec-ade9-a81da6375017 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:42.196673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 727/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:44.277022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.277565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.277565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.278936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:44.284038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414d0ae-d6d4-4796-8e6d-f91bfce64c99 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.285117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414d0ae-d6d4-4796-8e6d-f91bfce64c99 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.285544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414d0ae-d6d4-4796-8e6d-f91bfce64c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.354294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b414d0ae-d6d4-4796-8e6d-f91bfce64c99 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:44.355539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b414d0ae-d6d4-4796-8e6d-f91bfce64c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:44.368724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.452806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:44.738709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:44.740287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:44.844809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d482dfe0-a487-4d72-b4ec-dbe5af850341 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:44.856863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:44.917211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:44.917812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:44.918094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:44.918360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:44.919987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:45.027633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:45.029072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b03634-4b1a-4610-ae9d-2ed4b70bc0b5 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:45.047462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 718/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:47.440709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:47.441286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:47.441286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:47.442605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:47.448055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1ab1-7769-41bb-bfad-a73c4b5d1e26 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:47.448486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1ab1-7769-41bb-bfad-a73c4b5d1e26 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:47.448629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af1ab1-7769-41bb-bfad-a73c4b5d1e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:47.498424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:47.517101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25af1ab1-7769-41bb-bfad-a73c4b5d1e26 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:47.518326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25af1ab1-7769-41bb-bfad-a73c4b5d1e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:47.615680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:47.901204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:47.902737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:48.005478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b6889-52c6-487d-a128-487f25ff3272 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:48.044608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:48.077817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:48.078420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:48.078704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:48.078972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:48.080621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:48.188079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:48.189472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52bdb8f-3ab8-467b-9714-a4ea39d2ff5a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:48.208365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 730/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:50.243063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.243748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.243748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.245023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:50.250342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11197-384d-4f38-863e-bbf57ade38a4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.250773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11197-384d-4f38-863e-bbf57ade38a4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.250880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11197-384d-4f38-863e-bbf57ade38a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.318435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-eff11197-384d-4f38-863e-bbf57ade38a4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:50.319663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eff11197-384d-4f38-863e-bbf57ade38a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.339840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.415070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:50.705026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:50.708220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.813760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efe493-489c-478a-b5da-44dfaeb126f2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:50.834038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:50.886491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:50.887084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:50.887384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:50.887650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:50.889501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:50.997499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:50.998854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c892c48-b9c1-4254-b836-6873c438cd68 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:51.016963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 721/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:52.979977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.980518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.980518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:52.981856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:52.987289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960c382-1122-4310-ad67-75d77022f137 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:52.987736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960c382-1122-4310-ad67-75d77022f137 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:52.987857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960c382-1122-4310-ad67-75d77022f137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.050555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.053371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6960c382-1122-4310-ad67-75d77022f137 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:53.054576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6960c382-1122-4310-ad67-75d77022f137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:53.150976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:53.435859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:53.437382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:53.548006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e47d90-771f-4ed2-beb1-13cbe002565c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:53.598135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:53.622774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:53.623395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:53.623669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:53.623932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:53.625509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:53.731465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:53.732885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7315dc37-5394-4bc6-8c66-4512b1325d60 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:53.754804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 733/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:55.750519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.751082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.751082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.752960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:26:55.758102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5d65-f4cf-4803-ac73-c125e0093988 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:55.758559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5d65-f4cf-4803-ac73-c125e0093988 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:55.758679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a5d65-f4cf-4803-ac73-c125e0093988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:55.824874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-789a5d65-f4cf-4803-ac73-c125e0093988 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:55.826085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-789a5d65-f4cf-4803-ac73-c125e0093988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:55.856149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:55.923718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:56.206810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:56.208383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.312640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca519a7-b6e4-4bd4-949e-c76db202bc5d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:56.342500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:56.387629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:56.388233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:56.388509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:56.388776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:56.390363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:56.498765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:56.500363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a40130-ce34-47ac-bef3-c57e5096b246 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:56.519428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 724/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:26:58.511618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.512146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.512146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.513325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:26:58.519062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17cc8-38ec-429f-be90-ece775be0389 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:58.519627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17cc8-38ec-429f-be90-ece775be0389 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:58.519782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e17cc8-38ec-429f-be90-ece775be0389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:58.604988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c7e17cc8-38ec-429f-be90-ece775be0389 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:26:58.606234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7e17cc8-38ec-429f-be90-ece775be0389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:58.616059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:58.718316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:26:59.030355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:26:59.033322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:59.138947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19da0b9-69a0-43a1-81aa-4cfe322ae36d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:26:59.196053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:26:59.214834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:26:59.215472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:26:59.215747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:26:59.216012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:26:59.217606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:26:59.332969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:26:59.334320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9158681-5b45-4966-ba1d-6f396da60918 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:26:59.353169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 736/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:26:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:01.321260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:01.321793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:01.321793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:01.323212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:01.328403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381777-8655-4479-8b1c-a30a0b0d15f9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:01.328863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381777-8655-4479-8b1c-a30a0b0d15f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:01.329465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3381777-8655-4479-8b1c-a30a0b0d15f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:01.399833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a3381777-8655-4479-8b1c-a30a0b0d15f9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:01.401139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3381777-8655-4479-8b1c-a30a0b0d15f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:01.444797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:01.510540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:01.792417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:01.793988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:01.906334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fff624-5d6b-43b5-8cc4-d9fd0d182e7d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:01.944904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:01.982018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:01.982626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:01.982901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:01.983190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:01.984852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:02.093955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:02.095891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f0590-6680-4dd7-b1ba-523e3f507aed service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:02.114285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 727/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:04.057311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:04.057882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:04.057882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:04.059195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:04.064360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee0c4-ab8b-40f3-a6ed-225f8c76ac9e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:04.064771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee0c4-ab8b-40f3-a6ed-225f8c76ac9e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:04.064916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ee0c4-ab8b-40f3-a6ed-225f8c76ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:04.130632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8d4ee0c4-ab8b-40f3-a6ed-225f8c76ac9e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:04.131885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8d4ee0c4-ab8b-40f3-a6ed-225f8c76ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:04.150612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:04.229678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:04.514379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:04.515954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:04.627520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c03a50-cd6e-4a70-aea5-c5e92ca7a5eb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:04.695100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:04.701688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:04.704955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:04.704955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:04.704955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:04.706155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:04.815651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:04.819493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4240f6f8-b7bf-4e83-bb9f-cc73b320ba97 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:04.835427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 739/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:06.752961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.753617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.753617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.754880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:06.760146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f375e76-c227-4a08-b460-7bea3d9c6c9d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:06.760570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f375e76-c227-4a08-b460-7bea3d9c6c9d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:06.760690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f375e76-c227-4a08-b460-7bea3d9c6c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:06.827095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6f375e76-c227-4a08-b460-7bea3d9c6c9d None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:06.828367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f375e76-c227-4a08-b460-7bea3d9c6c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:06.876700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:06.927009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:07.216547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:07.218091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.323876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bfc63-c6bd-4393-8d87-01a98e64f6c1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:07.372398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:07.399615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:07.400248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:07.400525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:07.400795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:07.402387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:07.512232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:07.513122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2860c12-4626-4242-afbe-d7abdd90eaca service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:07.532280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 730/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:16.925239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=22 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:17.529103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=66 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:27.236865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948258e-9ad2-4ec8-9cfa-0a22afaf54af None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:27.237444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948258e-9ad2-4ec8-9cfa-0a22afaf54af None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:27.237444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948258e-9ad2-4ec8-9cfa-0a22afaf54af None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:27.238729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 741/1471] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:27:27.243911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832d31-fc09-492c-9a1d-82d82193c5bf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:27.244368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832d31-fc09-492c-9a1d-82d82193c5bf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:27.244468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7832d31-fc09-492c-9a1d-82d82193c5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:27.310743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7832d31-fc09-492c-9a1d-82d82193c5bf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:27.312006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7832d31-fc09-492c-9a1d-82d82193c5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:27.409659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 731/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:27:27.708645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:27.708926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:27.709197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:27.710513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 742/1473] 10.222.0.16 () {54 vars in 967 bytes} [Thu Sep 1 16:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:27:27.715442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cf71-e4d5-4da3-bafc-fc8e9107d91d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:27.715728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cf71-e4d5-4da3-bafc-fc8e9107d91d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:27.716001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cf71-e4d5-4da3-bafc-fc8e9107d91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:27.791206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-32e4cf71-e4d5-4da3-bafc-fc8e9107d91d None None] MFA Rules not processed for user `e346b0a022ea4218a4bd10c7e6fe6d91`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:27.792406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-32e4cf71-e4d5-4da3-bafc-fc8e9107d91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:27.812840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:27.867405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 732/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Thu Sep 1 16:27:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:27.875346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:27.876870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:27.988762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457f363-d2da-40f6-9704-7f44f8e25533 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:28.061428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:28.062029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:28.062309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:28.062580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:28.064217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:28.118717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:28.174066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:28.175541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f5e6c-58a4-46ce-85d1-54b8e6f21789 service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:28.194665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 743/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:30.220217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:30.220806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:30.220806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:30.222082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 733/1476] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:30.227259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f95e-e38e-4c6a-89d3-3cd4001000d7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:30.227670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f95e-e38e-4c6a-89d3-3cd4001000d7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:30.227810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f95e-e38e-4c6a-89d3-3cd4001000d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:30.294091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f9e6f95e-e38e-4c6a-89d3-3cd4001000d7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:30.296158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9e6f95e-e38e-4c6a-89d3-3cd4001000d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:30.394054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 744/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:30.672077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:30.673606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:30.799281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cfb5e-c989-4204-9102-cc6e1a84ca06 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:30.848740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:30.893991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:30.894588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:30.894858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:30.895124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:30.895785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:30.897508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:27:30.897508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:27:30.897508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:27:30.898639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca929ccf-bc6f-4a87-86ec-3fe011baab24 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:30.917802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 734/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Sep 1 16:27:30 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:27:32.855647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:32.856211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:32.856211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:32.858300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 745/1479] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:32.863767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66004-a544-4604-a1f8-3d42476040b6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:32.864181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66004-a544-4604-a1f8-3d42476040b6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:32.864298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be66004-a544-4604-a1f8-3d42476040b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:32.933549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7be66004-a544-4604-a1f8-3d42476040b6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:32.934754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7be66004-a544-4604-a1f8-3d42476040b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:33.031779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 735/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:33.333584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:33.335170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:33.345615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:33.482756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b578a88-f16e-4a15-b7c4-7c47b482a107 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:33.560170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:33.593052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:33.593646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:33.593917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:33.594538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:33.596160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:33.683230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:33.684841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2731849-2915-4239-80dd-7735bf6a8017 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:33.703943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 746/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:35.802968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:35.803577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:35.803577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:35.804924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 736/1482] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:35.810614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e946-708b-4132-9b13-ac3119b24279 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:35.811239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e946-708b-4132-9b13-ac3119b24279 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:35.811239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e946-708b-4132-9b13-ac3119b24279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:35.877986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3e47e946-708b-4132-9b13-ac3119b24279 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:35.879209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3e47e946-708b-4132-9b13-ac3119b24279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:35.924501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:35.975252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 747/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:36.260797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:36.262476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:36.286640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:36.366169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04c0f7-d8fd-4f6d-a4ae-1f287f58669b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:36.441929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:36.442539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:36.442817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:36.443087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:36.444733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:36.527527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:36.565986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:36.567758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ad885-7bb0-48a1-a4f5-28b51c16a8e3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:36.586409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 737/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:38.557732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:38.558289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:38.558289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:38.559720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 748/1485] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:38.565491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aaf8e-c3f8-4151-8299-01ef441bbf28 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:38.565934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aaf8e-c3f8-4151-8299-01ef441bbf28 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:38.566043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34aaf8e-c3f8-4151-8299-01ef441bbf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:38.631764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d34aaf8e-c3f8-4151-8299-01ef441bbf28 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:38.632969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d34aaf8e-c3f8-4151-8299-01ef441bbf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:38.728504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 738/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:39.024579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:39.026114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:39.130046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273237d0-b88d-427a-a0f0-2d010508ef2b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:39.200715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:39.202493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:39.203093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:39.203398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:39.203759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:39.205267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:39.320370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:39.321832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6792a-6834-443e-9f2f-83b2c0dd3f09 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:39.343463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 749/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:41.350518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928aa20-1083-4608-8175-11982b1b344e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.351083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928aa20-1083-4608-8175-11982b1b344e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.351083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928aa20-1083-4608-8175-11982b1b344e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.352421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 739/1488] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:41.358304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f973a-715f-4c09-83a0-5028e0749069 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.360247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f973a-715f-4c09-83a0-5028e0749069 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.360247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f973a-715f-4c09-83a0-5028e0749069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.425326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-494f973a-715f-4c09-83a0-5028e0749069 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:41.426543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-494f973a-715f-4c09-83a0-5028e0749069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:41.477932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:41.521234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 750/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:41.812313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928aa20-1083-4608-8175-11982b1b344e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:41.813873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6928aa20-1083-4608-8175-11982b1b344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:41.850665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:41.918010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928aa20-1083-4608-8175-11982b1b344e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:41.993536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:41.994125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:41.994398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:41.994665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:41.996278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:42.078930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:42.109089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:42.110537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ee578-cd45-480f-85aa-91e5322a9a4e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:42.128854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 740/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:44.439222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:44.439769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:44.439769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:44.441135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 751/1491] 10.222.0.16 () {54 vars in 927 bytes} [Thu Sep 1 16:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:27:44.446401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f0d0-16a6-4f6b-bcc9-cad6d5f18a62 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:44.446840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f0d0-16a6-4f6b-bcc9-cad6d5f18a62 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:44.446951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96f0d0-16a6-4f6b-bcc9-cad6d5f18a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:44.513250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6d96f0d0-16a6-4f6b-bcc9-cad6d5f18a62 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:27:44.514382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d96f0d0-16a6-4f6b-bcc9-cad6d5f18a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:44.610084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 741/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Thu Sep 1 16:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:27:44.900972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:27:44.902512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:45.005626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8985b5-0443-4dbb-aac0-0feb3762b549 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:27:45.079826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:27:45.080425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:27:45.080701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:27:45.080966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:27:45.082564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:27:45.091576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:45.191108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:27:45.192492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a06e2-b955-4490-92a0-636b79205e17 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:27:45.211004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 752/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:27:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:27:54.608188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=82 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:27:55.208759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=50 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:08.520538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:08.521102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:08.521102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:08.596437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:08.597635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:08.694315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 742/1494] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:08.702689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:08.703259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:08.703259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:08.779859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:08.781024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:08.879039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 753/1495] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:08.883953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:08.885190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:08.896541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b43ad3f-673b-4297-a084-4aeea72dcf55 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:08.961235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:08.995461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:08.996093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:08.996373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:08.996645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:08.997288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:29:08.997588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:08.998606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:08.998606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:29:08.998606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:29:08.999587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.011544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 743/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:29:09.020152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.030485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b2abe-8aa8-43d0-97ba-170bbec1b8a9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.092156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:09.125671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.126469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.126983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.127577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.128214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:09.129914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:09.129914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:29:09.129914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:29:09.131534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.270735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 754/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:29:09 2022] POST /identity/v3/projects => generated 379 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:29:09.275446 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.276959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.310464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6565e-4672-4b14-b9f4-0b0a5c25ac17 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.411387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.412010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.412519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.412787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.413425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:09.414447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:09.414447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:29:09.414447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:29:09.415721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.543721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 744/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:29:09 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:29:09.547894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.549441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.611876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607415e0-62c8-499d-8377-928639bf977a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.697245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:09.708634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.709284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.709567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.709836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.710478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:09.711523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:09.711523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:29:09.711523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:29:09.712413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.725523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 755/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:29:09.729281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.730877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.742938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e3146-f65f-4fe0-bdaa-a745268f1d96 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.768054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:09.838269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.838869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.839172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.839442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:09.840074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:09.841086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:09.841086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:29:09.841086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:29:09.841968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59d0ca-74b2-4158-97ba-8da27ef6fdd6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:09.855996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 745/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:29:09.860258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:09.861780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:09.895600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77a48-fd88-4ea5-a898-0fb5b575267d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:09.989210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:09.989856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:09.990159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:09.990540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] PATH_INFO: `/v3/projects/a8f5fd332be544a9a87592d577db727d/users/b4a9cf8bd6c9476d966541757b3beea5/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.012249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f5fd332be544a9a87592d577db727d, user_id=b4a9cf8bd6c9476d966541757b3beea5, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:10.013265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:10.013265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:29:10.013265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:29:10.015229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e63da5e-1c70-460e-b8ba-27d07646ddc8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:10.114884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 756/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:29:09 2022] PUT /identity/v3/projects/a8f5fd332be544a9a87592d577db727d/users/b4a9cf8bd6c9476d966541757b3beea5/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:29:10.127319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec452-3330-40e9-bbc7-154128cd401b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.127631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec452-3330-40e9-bbc7-154128cd401b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.127904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec452-3330-40e9-bbc7-154128cd401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.185295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-42dec452-3330-40e9-bbc7-154128cd401b None None] MFA Rules not processed for user `b4a9cf8bd6c9476d966541757b3beea5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:10.186683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-42dec452-3330-40e9-bbc7-154128cd401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.265786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 746/1502] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:10.279523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.279805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.280078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.281399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 757/1503] 10.222.0.16 () {54 vars in 964 bytes} [Thu Sep 1 16:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:10.286014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bc7a-1cd6-4b0f-95d7-44ad2aac5f1b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.286298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bc7a-1cd6-4b0f-95d7-44ad2aac5f1b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.286571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0bc7a-1cd6-4b0f-95d7-44ad2aac5f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.316908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:10.359832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-07d0bc7a-1cd6-4b0f-95d7-44ad2aac5f1b None None] MFA Rules not processed for user `4a081c08049447949477e73e0891f1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:10.361035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-07d0bc7a-1cd6-4b0f-95d7-44ad2aac5f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.438407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 747/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Sep 1 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:10.448437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:10.449957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.499759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:10.553226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4df95-9d1b-4b6f-b196-5d55a898b74f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:10.626030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:10.626657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.626937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.627237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.628841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.727019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:10.728430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:10.731940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:10.747737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 758/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:10.793207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b369ac-44ed-4bd8-b982-c227d52e3d38 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.793635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b369ac-44ed-4bd8-b982-c227d52e3d38 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.793744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b369ac-44ed-4bd8-b982-c227d52e3d38 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:10.795002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 748/1506] 10.222.0.16 () {52 vars in 830 bytes} [Thu Sep 1 16:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:29:10.798491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:10.799740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:10.898369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0db224-8517-4487-9387-13b55341a74d service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:10.990384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:10.991118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:10.991426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:10.991697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] PATH_INFO: `/v3/users/b4a9cf8bd6c9476d966541757b3beea5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:11.003700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4a9cf8bd6c9476d966541757b3beea5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:11.004713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:11.004713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:29:11.004713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:29:11.006512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:11.019819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:11.226223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4a9cf8bd6c9476d966541757b3beea5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:11.297114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4a9cf8bd6c9476d966541757b3beea5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:11.297667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4a9cf8bd6c9476d966541757b3beea5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:11.321480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4a9cf8bd6c9476d966541757b3beea5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:11.321913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4a9cf8bd6c9476d966541757b3beea5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:11.349369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 759/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:29:10 2022] DELETE /identity/v3/users/b4a9cf8bd6c9476d966541757b3beea5 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:29:11.354347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:11.354633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:11.354904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:11.410992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:29:11.413067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:11.487547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:11.500973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 749/1508] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:29:11.510108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:11.511361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:11.615324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8fa534-9e5a-453e-a3d0-6171628404b4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:11.668907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:11.688436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:11.689044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:11.689320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:11.689594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:11.691360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:11.800981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:11.802540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07788a64-8e95-4fbc-971a-22b99cfb7cec service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:11.821475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 760/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:29:12.453502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:29:12.454753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:29:12.466856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8279bd70-8900-4922-add1-5ec274c37352 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:29:12.565406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:29:12.566041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:29:12.566324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:29:12.566594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] PATH_INFO: `/v3/projects/a8f5fd332be544a9a87592d577db727d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:29:12.577526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f5fd332be544a9a87592d577db727d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:29:12.578561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:29:12.578561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:29:12.578561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:29:12.579874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:29:12.709598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f5fd332be544a9a87592d577db727d', 'request_id': 'req-36b0c81d-1037-4d82-9c7a-835b3e43cbee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:12.727187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f5fd332be544a9a87592d577db727d', 'request_id': 'req-36b0c81d-1037-4d82-9c7a-835b3e43cbee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:12.794533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f5fd332be544a9a87592d577db727d', 'request_id': 'req-36b0c81d-1037-4d82-9c7a-835b3e43cbee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:12.819256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] The token cache is being invalidate because project a8f5fd332be544a9a87592d577db727d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:29:12.819540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:12.820037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-36b0c81d-1037-4d82-9c7a-835b3e43cbee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:29:12.821442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 750/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:29:12 2022] DELETE /identity/v3/projects/a8f5fd332be544a9a87592d577db727d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:29:21.819205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=62 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:29:22.816548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:47.696550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:47.697164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:47.697164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:47.698518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:47.703476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824d62-e94f-40cc-bf16-0fce95fba87b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:47.703896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824d62-e94f-40cc-bf16-0fce95fba87b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:47.704079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83824d62-e94f-40cc-bf16-0fce95fba87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:47.789857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83824d62-e94f-40cc-bf16-0fce95fba87b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:47.791199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83824d62-e94f-40cc-bf16-0fce95fba87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:47.900221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:48.056483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:48.058619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:48.180652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19fdc-cfe8-4b94-b63d-1cfff0d3306a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:48.186650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d01496-c201-4035-82a5-843483032627 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:48.187291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01496-c201-4035-82a5-843483032627 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:48.187575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01496-c201-4035-82a5-843483032627 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:48.187851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d01496-c201-4035-82a5-843483032627 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:48.196935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d01496-c201-4035-82a5-843483032627 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:48.198525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d01496-c201-4035-82a5-843483032627 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:48.208200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 762/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:49.051801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28577e8-3c2b-4da1-8d9e-16206e07beb7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:49.052374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28577e8-3c2b-4da1-8d9e-16206e07beb7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:49.052374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28577e8-3c2b-4da1-8d9e-16206e07beb7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:49.053715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:30:49.059627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7320-33ba-4b82-bf69-d99ef60bc580 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:49.060418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7320-33ba-4b82-bf69-d99ef60bc580 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:49.060513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d7320-33ba-4b82-bf69-d99ef60bc580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:49.080457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4a8d7320-33ba-4b82-bf69-d99ef60bc580 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:49.081676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a8d7320-33ba-4b82-bf69-d99ef60bc580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:49.089685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:30:49.244472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28577e8-3c2b-4da1-8d9e-16206e07beb7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:49.259380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28577e8-3c2b-4da1-8d9e-16206e07beb7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:49.264902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:49.265512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:49.265787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:49.266056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:49.275101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:49.276778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655cf63-c79b-4ba8-b067-7345db37a838 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:49.286760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 753/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:30:50.003507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eda17-e443-4f35-a3b9-4e04f2fd1531 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.003793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eda17-e443-4f35-a3b9-4e04f2fd1531 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.004097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63eda17-e443-4f35-a3b9-4e04f2fd1531 None None] PATH_INFO: `/v3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:50.005364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 764/1517] 10.222.0.42 () {54 vars in 925 bytes} [Thu Sep 1 16:30:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:30:50.012228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08f32-5af1-4751-8db3-781b805ddadd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:50.012518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08f32-5af1-4751-8db3-781b805ddadd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:50.012794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c08f32-5af1-4751-8db3-781b805ddadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:50.073645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-48c08f32-5af1-4751-8db3-781b805ddadd None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:50.074866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48c08f32-5af1-4751-8db3-781b805ddadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:50.092861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:50.149885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 754/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 1 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:59.075313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=56 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:30:59.724581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef68981-6c59-4657-9ec8-b60bf02146d6 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.724829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef68981-6c59-4657-9ec8-b60bf02146d6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.725050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef68981-6c59-4657-9ec8-b60bf02146d6 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.727988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 765/1519] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 1 16:30:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:30:59.733403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766a70-cb04-4fa4-b0f1-72321550b38f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.733903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766a70-cb04-4fa4-b0f1-72321550b38f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.734002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766a70-cb04-4fa4-b0f1-72321550b38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.754239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-85766a70-cb04-4fa4-b0f1-72321550b38f None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:30:59.758509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-85766a70-cb04-4fa4-b0f1-72321550b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:30:59.765155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 755/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 1 16:30:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:30:59.772758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef68981-6c59-4657-9ec8-b60bf02146d6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:30:59.790222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef68981-6c59-4657-9ec8-b60bf02146d6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:30:59.795782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:30:59.796669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:30:59.797041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:30:59.797211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:30:59.809339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:30:59.811767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54247f39-4c04-4102-ae82-069a00b8c23b service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:30:59.825254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 766/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:30:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:31:00.546106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27046b-9019-4c1c-8c93-beb2a9ee556d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.546390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27046b-9019-4c1c-8c93-beb2a9ee556d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.546764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27046b-9019-4c1c-8c93-beb2a9ee556d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.548201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:31:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:00.554125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c6398-b0fb-4a29-be1a-9984bc2841d1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.555224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c6398-b0fb-4a29-be1a-9984bc2841d1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.555991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c6398-b0fb-4a29-be1a-9984bc2841d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.580812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-853c6398-b0fb-4a29-be1a-9984bc2841d1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:00.582048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-853c6398-b0fb-4a29-be1a-9984bc2841d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:00.590782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:00.750181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27046b-9019-4c1c-8c93-beb2a9ee556d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:00.767386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27046b-9019-4c1c-8c93-beb2a9ee556d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:00.774418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:00.774744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:00.775023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:00.775344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:00.784516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:00.786223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0953c11-868e-46b8-abe8-aba837339482 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:00.796770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 757/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:01.647646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f1bf3-57a3-4291-af37-f627999a077c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.648836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f1bf3-57a3-4291-af37-f627999a077c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.648836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f1bf3-57a3-4291-af37-f627999a077c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.650578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 1 16:31:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:01.655556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd1ef3-4598-4f14-bf19-2848100256e0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.655838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd1ef3-4598-4f14-bf19-2848100256e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.656107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd1ef3-4598-4f14-bf19-2848100256e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.676922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-56fd1ef3-4598-4f14-bf19-2848100256e0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:01.678136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56fd1ef3-4598-4f14-bf19-2848100256e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:01.686751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 1 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:01.842098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f1bf3-57a3-4291-af37-f627999a077c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:01.856919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f1bf3-57a3-4291-af37-f627999a077c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:01.862499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:01.863097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:01.863407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:01.863684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:01.872235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:01.873814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa308b3-c53f-400e-a10c-6adc85f791a0 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:01.883724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 769/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:06.897254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.897800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.897800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.899677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:06.905656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539981-2291-40e6-a172-c98a4d9eda0f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:06.906168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539981-2291-40e6-a172-c98a4d9eda0f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:06.906275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00539981-2291-40e6-a172-c98a4d9eda0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:06.926723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-00539981-2291-40e6-a172-c98a4d9eda0f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:06.927975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-00539981-2291-40e6-a172-c98a4d9eda0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:06.936244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:07.233444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:07.235869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:07.322149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9452f-e86d-4efd-aa06-60e1223080d1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:07.325980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:07.326551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:07.326831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:07.327185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:07.337669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:07.339213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57f6c4-87a5-4c15-8377-03beb71eee19 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:07.349023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 760/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:09.324739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7d22d-bdf8-40e1-81b3-dcb864cdb3f2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.325691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7d22d-bdf8-40e1-81b3-dcb864cdb3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.325691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7d22d-bdf8-40e1-81b3-dcb864cdb3f2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.326614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:31:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:09.332233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af053b6-3e68-4a95-8970-6f1c1a92d7f1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.332759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af053b6-3e68-4a95-8970-6f1c1a92d7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.332759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af053b6-3e68-4a95-8970-6f1c1a92d7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.353049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1af053b6-3e68-4a95-8970-6f1c1a92d7f1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:09.354272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1af053b6-3e68-4a95-8970-6f1c1a92d7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:09.362565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:31:09.660691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7d22d-bdf8-40e1-81b3-dcb864cdb3f2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:09.673625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7d22d-bdf8-40e1-81b3-dcb864cdb3f2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:09.677912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:09.678637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:09.678984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:09.679902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:09.692152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:09.693932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf5747-a603-454b-b42d-c4a11a549be5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:09.707008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 772/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:13.473072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.473620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.473620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.474910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:13.481569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bd55-122d-4d71-95cb-d82089e2618a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:13.482359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bd55-122d-4d71-95cb-d82089e2618a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:13.482359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98bd55-122d-4d71-95cb-d82089e2618a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:13.502516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cc98bd55-122d-4d71-95cb-d82089e2618a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:13.503770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cc98bd55-122d-4d71-95cb-d82089e2618a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:13.511912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:14.095765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:14.097581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:14.171907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:14.186925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898cec28-5f59-416f-8de1-e769b5d087bc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:14.190612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:14.191252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:14.191579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:14.191797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:14.202188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:14.204279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b2d1f-6cc2-47f7-a65c-149328a4fd38 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:14.214561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 763/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:16.234228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.234881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.234881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.236160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 1 16:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:31:16.242093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ed5c-da27-4558-9930-36b20cac2fe5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.242656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ed5c-da27-4558-9930-36b20cac2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.242656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8ed5c-da27-4558-9930-36b20cac2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.265080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-91b8ed5c-da27-4558-9930-36b20cac2fe5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:31:16.266376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91b8ed5c-da27-4558-9930-36b20cac2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:16.275049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 1 16:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:31:16.844464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:31:16.846151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:31:16.862389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c3e2e-1b46-4929-a1a5-883d96194803 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:31:16.868355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:31:16.868961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:31:16.869250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:31:16.869500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:31:16.881626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:31:16.883199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa1fd92-5611-46d3-bad8-cd526a1d10bb service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:31:16.893611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 775/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:31:26.259096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=11 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:31:26.889094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=33 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:46.284178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.284868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.284868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.320980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.323570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.341614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 765/1540] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.350790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdabd-5d70-45fe-a70e-830fb81ca2a6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.351533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdabd-5d70-45fe-a70e-830fb81ca2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.351533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88cdabd-5d70-45fe-a70e-830fb81ca2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.388060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d88cdabd-5d70-45fe-a70e-830fb81ca2a6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.389948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d88cdabd-5d70-45fe-a70e-830fb81ca2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.406319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 776/1541] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.411193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.423698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963d149-a246-4a64-8a7f-fe9ababa8514 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.433676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.434309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.436264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f546d4e-a2ae-43a9-8d76-6adf51eb8e17 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.436542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f546d4e-a2ae-43a9-8d76-6adf51eb8e17 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.437080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f546d4e-a2ae-43a9-8d76-6adf51eb8e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.440428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.440738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.441403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:46.441784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.449238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e1f765-afa9-40f9-8b92-e6d5a71659c4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.477121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2f546d4e-a2ae-43a9-8d76-6adf51eb8e17 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.478309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2f546d4e-a2ae-43a9-8d76-6adf51eb8e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.483179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 766/1542] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:46.485448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.485742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.486020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.510439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 777/1543] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.512812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540322-38dd-4872-9dc7-ee9073e3c6f1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.513187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540322-38dd-4872-9dc7-ee9073e3c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.513371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7540322-38dd-4872-9dc7-ee9073e3c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.548338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.549850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.563369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c7540322-38dd-4872-9dc7-ee9073e3c6f1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.564750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c7540322-38dd-4872-9dc7-ee9073e3c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.574965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 767/1544] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.576988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.589970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 778/1545] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.591993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab7157-81de-455b-a95a-6b2b9050a141 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.592617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab7157-81de-455b-a95a-6b2b9050a141 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.593098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fab7157-81de-455b-a95a-6b2b9050a141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.611433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0e563-fac5-4345-b228-5e0ab4619945 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.617428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.618055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.618338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.618613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.619294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.619732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7fab7157-81de-455b-a95a-6b2b9050a141 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.620341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:46.620341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:32:46.620341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:32:46.620975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7fab7157-81de-455b-a95a-6b2b9050a141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.621555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.631657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 779/1546] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.633754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d1f4-3a73-41b2-8234-ba9186970490 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.634039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d1f4-3a73-41b2-8234-ba9186970490 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.634612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22d1f4-3a73-41b2-8234-ba9186970490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.669572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cc22d1f4-3a73-41b2-8234-ba9186970490 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.670807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cc22d1f4-3a73-41b2-8234-ba9186970490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.681084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 780/1547] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.692761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ad32-c36c-43d2-bad5-f8c088c86960 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.692761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ad32-c36c-43d2-bad5-f8c088c86960 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.692761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ad32-c36c-43d2-bad5-f8c088c86960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.707045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f318ad32-c36c-43d2-bad5-f8c088c86960 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.708296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f318ad32-c36c-43d2-bad5-f8c088c86960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.716967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 781/1548] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.718272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2f0-9391-45d7-ab85-abadda87c611 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.718556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2f0-9391-45d7-ab85-abadda87c611 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.718822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634db2f0-9391-45d7-ab85-abadda87c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.743048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-634db2f0-9391-45d7-ab85-abadda87c611 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.744317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-634db2f0-9391-45d7-ab85-abadda87c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.752560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 782/1549] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.753788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce694-61d8-4780-8286-593fd1d04c40 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.754070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce694-61d8-4780-8286-593fd1d04c40 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.754374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce694-61d8-4780-8286-593fd1d04c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.766350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 768/1550] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:46.767312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.779261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-459ce694-61d8-4780-8286-593fd1d04c40 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.780576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-459ce694-61d8-4780-8286-593fd1d04c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.857663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b16b52-caa4-412f-9ed3-bb9a1789adda admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.862614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.863348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.863624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.863891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.864542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:46.864890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.872856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 783/1551] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.873073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5917739-9e10-4d9a-8e82-2b8fa4049784 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.873176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47687606-d052-48d8-9f90-0b0693b1508c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.873176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47687606-d052-48d8-9f90-0b0693b1508c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.873176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47687606-d052-48d8-9f90-0b0693b1508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.883713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 769/1552] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:46.886976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.887571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.893296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.893504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-47687606-d052-48d8-9f90-0b0693b1508c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.895113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-47687606-d052-48d8-9f90-0b0693b1508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.908304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 784/1553] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.909583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.909876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.910154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.914710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.916052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.924650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 770/1554] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.926845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.933071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:46.947972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:46.948135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27903bb3-a21a-4083-b3a7-7f02debd1ae9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.948256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 785/1555] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:46.948256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.948438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.948438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.948438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.948438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.948843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:46.948843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.948843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.958674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e4e03-54d2-4610-aad2-dbd24fb1d4e8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.965740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 771/1556] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:46.965740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:46.969654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.970493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.970881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.971443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.972255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:46.972596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.973736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:46.973736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:32:46.973736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:32:46.975130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d9791-9632-4d5c-b128-a6e2f501c0ce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:46.975695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac069bd-92be-401e-b3f1-87ac66467f0a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.980447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:46.981262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.981545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.981967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:46.982680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:46.984813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da100d57-c10e-4fa2-8b74-a49dee23ee64 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:46.989426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 786/1557] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:46.991777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:46.992065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:46.992340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.014273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:47.019099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:47.024692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 787/1558] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:47.025525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.035462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25a6f7-4eea-4354-9014-f444ec6e426b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.041493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.041712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.041881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.042178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.042883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.044921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.096581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 772/1559] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:46 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.097866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10f3b-e0ad-4b30-a783-502acd33b21f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.098184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10f3b-e0ad-4b30-a783-502acd33b21f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.098437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10f3b-e0ad-4b30-a783-502acd33b21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.127755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-20d10f3b-e0ad-4b30-a783-502acd33b21f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:47.127755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20d10f3b-e0ad-4b30-a783-502acd33b21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:47.131699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 773/1560] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:47.132876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.133156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.133427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.154113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:47.155378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:47.165075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 774/1561] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:47.166035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.176380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4647405-9151-42a2-b332-bbabebf6b5ed None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.181154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.181745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.182018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.182284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.182960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:47.183292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.184903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.198926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 775/1562] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:47.199810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.210132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 788/1563] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/projects => generated 393 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.210132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.213361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f367a8a-d8b6-4804-be2a-3fb05d5029ed admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.255290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:47.293504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ff28-654a-4639-b3d5-78434ca2df40 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.294213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.294870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.295167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.295445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.296087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.298119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.298521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.299986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.299986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.299986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.300474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.302469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.307379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:47.473877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 776/1564] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/projects => generated 379 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.475530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.506800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 789/1565] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/projects => generated 377 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.508233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.558066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd5607-419f-4c83-a982-2cf73ce8253a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.578367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e3087-4254-4afe-b355-7db0bd4b805b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.585317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.585944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.586221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.586491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.587198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.588806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.602879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 790/1566] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:47.610066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.618567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e8e11-6bfa-4086-8306-5df35d7504e5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.623651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.624484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.624956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.625486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.626669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:47.627124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.628796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.633659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.639997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.649716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 791/1567] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:47.649716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.656514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 777/1568] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:47.656919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca236a4-0cac-4100-b693-a8fc9f131080 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.658127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.662573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.663300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.663674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.664016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.664782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:32:47.665157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.666898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.675074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745592e7-68ae-4d04-bd18-02f7dc23c811 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.680700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 792/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:32:47.681471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.681964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.682590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.682867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.683138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.683824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.685846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.695632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5f368-1ae2-408d-9e80-da82aee01fda admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.703507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.704326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.704790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.705183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.706059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.707431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:47.707431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:32:47.707431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:32:47.709944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.785272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 778/1570] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.786255 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.831127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 793/1571] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:47.832032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:47.845362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf8a71-21d2-44f4-986e-0449320d154c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.855843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:47.879856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c5f43-2e28-48e7-a20b-ffdb1a5966ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:47.885402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.886044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.886322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.886592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.887264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.889261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:47.919391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:47.920701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:47.920701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:47.920961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:47.922521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:47.923599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.055834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 779/1572] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.056599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.076168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f9411-70b1-4fb0-ac90-7916e6d8a4ce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.077345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 794/1573] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:47 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.078166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.087578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.088567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.089031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.089468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.090534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.093246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.104689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e99539-be94-40e2-bb41-2cc1cb037024 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.110476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.114323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 780/1574] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:48.114566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.114566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.114566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.114937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.115023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.115254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.123219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57263-6d34-4b50-8f44-c198694e6063 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.131528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.132266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.132549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.132812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.133534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.135606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.217757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 795/1575] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.218573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.224019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:48.257478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 781/1576] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.262963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.287115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e88-a2cd-4fd1-9ed0-7415f2ea79e6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.347447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.347961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.348152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fbb0e-6d79-40f4-930d-6cd86fdc502f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.348917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.348917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.349244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.351110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.354127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.354830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.355226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.355679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.356430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.358535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.497323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 796/1577] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/projects => generated 365 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.498160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.512400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 782/1578] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.513391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.525093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5346f-33ac-45fa-888c-3bef516a953f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.571105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04200a-c575-4058-aa4d-56b725c523ec admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.587428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.588059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.588341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.588627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.589253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.589536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.590157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.590433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.590870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.590973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.591567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.593170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.602655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 797/1579] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:48.606540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.606753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 783/1580] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:48.607669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.620207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ae7a2-1c9e-482a-991c-b66284533f0e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.620943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae90f08-12ed-4296-a7ef-224d5b83b25b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.625051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.625616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.625800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.625980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.626285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] PATH_INFO: `/v3/projects/a994270b8e1f43328b827696f9d89bb0/users/8a45ee95cf5947c49584ff403facaff7/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.626420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.626643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.626914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.627584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.629442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.643803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a994270b8e1f43328b827696f9d89bb0, user_id=8a45ee95cf5947c49584ff403facaff7, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.644823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 798/1581] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:48.644949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:32:48.644949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:32:48.644949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:32:48.645664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.647732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.652248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:48.657378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc6f1c-456b-4035-a827-901848ec2318 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.663488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.672670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.672670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.672670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.672670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.672670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.733471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 784/1582] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:48 2022] PUT /identity/v3/projects/a994270b8e1f43328b827696f9d89bb0/users/8a45ee95cf5947c49584ff403facaff7/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:48.734998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.767678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 799/1583] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.768947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.777748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:48.784174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67f6f4-892e-430e-ba2c-fb7a9594696b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.817083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b84864-7702-4e98-b2a0-e21ad3189c51 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.824820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.827723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.827723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.827723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.827723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.830900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.860685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.861542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.862035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.862460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.863335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.865784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:48.967975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 785/1584] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:48.968828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:48.980884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fcbcfd-08c0-4783-b223-16f08489129a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:48.985703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:48.986348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:48.986617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:48.986950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:48.987603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:48.989307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.004156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 786/1585] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.017853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.018079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 800/1586] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:48 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:49.018079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.020014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf0f48-1a63-476d-8f69-6e89be40cd4d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.023013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c85bb-2578-4ca2-86b8-0c23e388630f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.026759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.027593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.028029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.028405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.029169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.030562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.031183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa485eb-1735-42e7-9bc7-72016c714d71 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.031516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.031961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.032346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.033342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.035575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.050405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 801/1587] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.051354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.051874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 787/1588] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.055286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.055922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.056464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.066362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52fda-2a3f-481d-a64a-103847eef50d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.073283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.073986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.074379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.074767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.075678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.077441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.079276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:49.080551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:49.090866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 802/1589] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.091781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.099701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 788/1590] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:49.100904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.107247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7da36b-c07c-4cc6-b8c6-95c12de9baff admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.112493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.113043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.113324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.113554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed9e79-7384-4287-97ed-9e985f1edf8b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.113668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.114233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.116481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.120359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.121697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.122147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.122656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.123789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.126008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.132545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 803/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.133520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.145388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da92fe0-d911-4582-876b-f04d9fb7258d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.152242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 789/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.152242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.152560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.152682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.153005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.153349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.154069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.155982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.161303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ec5d2-20f0-4e08-8b94-728ef0032b46 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.167609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.168267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 804/1593] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.168434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.168654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.168905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.169117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.169620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.171372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.180267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb3d51-f052-4964-abde-b339b70350ca admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.184338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 790/1594] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.185391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.185597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.186053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.186358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.186960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] PATH_INFO: `/v3/projects/c504a04e6b84427b9732157524a4aeef/users/662f85837db7411e8695d971413fc5ba/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.201550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3f9cb-ab48-4091-9369-280ea68ed34f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.209674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.210476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.210800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.211120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] PATH_INFO: `/v3/projects/2f0958b1518e4439bf12959b7760abc4/users/8937e9f80f1540e389ee283971624741/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.211623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c504a04e6b84427b9732157524a4aeef, user_id=662f85837db7411e8695d971413fc5ba, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.215263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.230878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0958b1518e4439bf12959b7760abc4, user_id=8937e9f80f1540e389ee283971624741, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.234031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.319417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 805/1595] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/c504a04e6b84427b9732157524a4aeef/users/662f85837db7411e8695d971413fc5ba/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:49.320285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.351565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 791/1596] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/2f0958b1518e4439bf12959b7760abc4/users/8937e9f80f1540e389ee283971624741/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:49.353052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.401444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ff374-057c-46f9-88dd-e335de90c932 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.422644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd294205-ca14-42a0-bd14-0dbc3c698164 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.427078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.428492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.428775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.429061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.438270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:49.461639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:49.469651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.470618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.470987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.471305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] PATH_INFO: `/v3/projects/c787c684c86144c792d9afc457ac3212/users/21e6ff7908e84a5994a51be0cfc82414/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.488547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c787c684c86144c792d9afc457ac3212, user_id=21e6ff7908e84a5994a51be0cfc82414, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.491870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.503434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.505172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.515581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 792/1597] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:49.516479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.526871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0385c-e59b-4a18-8f9c-a3df5ba6517a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.533747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.534638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.535295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.535467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.536934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.539879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.550972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 793/1598] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.551874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.564879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402adece-2bdf-40df-bc54-6a44bad5ab68 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.570752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.571431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.571778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.575626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.575626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.575626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.585463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 794/1599] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.586364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.596838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3cbfa-1fdf-488e-a87f-f1b23b512675 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.604009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.604666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.605052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.605292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] PATH_INFO: `/v3/projects/b14e9793530a41bbab5bd4b3caecdb3c/users/ab4df1724851433889a519583a5333b3/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.625110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 806/1600] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/c787c684c86144c792d9afc457ac3212/users/21e6ff7908e84a5994a51be0cfc82414/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:49.626056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.627580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14e9793530a41bbab5bd4b3caecdb3c, user_id=ab4df1724851433889a519583a5333b3, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.632998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.706112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba12fac-b5b3-44d3-992f-7d3cc47b0b4c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.712640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.713417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.713704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.713969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.714623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.716474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1091000-5584-4424-8426-3a2f03911a5a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.734061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 807/1601] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:49.735361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93703ad-1d20-46ad-8d7a-8ce07dff799f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.736594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93703ad-1d20-46ad-8d7a-8ce07dff799f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.736594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93703ad-1d20-46ad-8d7a-8ce07dff799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.760311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b93703ad-1d20-46ad-8d7a-8ce07dff799f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:49.761697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b93703ad-1d20-46ad-8d7a-8ce07dff799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:49.775300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 808/1602] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:49.777010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 795/1603] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/b14e9793530a41bbab5bd4b3caecdb3c/users/ab4df1724851433889a519583a5333b3/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:49.777010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.777623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.778104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.778571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.807872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:49.809438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:49.851376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30509afc-1a87-4b0d-b097-41276bb2ae12 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.856245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.856963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.857238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.857511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] PATH_INFO: `/v3/projects/5133df6dc2af4bd0bf409bc7a6678ebc/users/9221e818e45c4fb0987c9527cee3868e/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.877554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 809/1604] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:49.878421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5133df6dc2af4bd0bf409bc7a6678ebc, user_id=9221e818e45c4fb0987c9527cee3868e, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.881570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:49.881748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42dd0ae-1556-46f2-99ca-afa0606a30d4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.894148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafdb4d1-3f6f-40c9-8e50-2290ac10214c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:49.900379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:49.901082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.901361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.901633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] PATH_INFO: `/v3/projects/762f35bfd1fa430ab8dcf86358ce0225/users/c2c50b74a2d34acd9656b40596682d13/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:49.918918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762f35bfd1fa430ab8dcf86358ce0225, user_id=c2c50b74a2d34acd9656b40596682d13, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:49.921690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:49.976703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 796/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/5133df6dc2af4bd0bf409bc7a6678ebc/users/9221e818e45c4fb0987c9527cee3868e/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:49.978112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2125587-017f-41bf-ac39-bd12deba25dc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:49.978112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2125587-017f-41bf-ac39-bd12deba25dc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:49.978334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2125587-017f-41bf-ac39-bd12deba25dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.004082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2125587-017f-41bf-ac39-bd12deba25dc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:50.006118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2125587-017f-41bf-ac39-bd12deba25dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.043773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 810/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:49 2022] PUT /identity/v3/projects/762f35bfd1fa430ab8dcf86358ce0225/users/c2c50b74a2d34acd9656b40596682d13/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:50.045743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.084753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:50.095414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:50.118462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 797/1607] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:50.119810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.120082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.120353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.132717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045306f-38ff-4250-aaa9-d549b6981156 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.138688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.140300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.140874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.141052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:50.141344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] PATH_INFO: `/v3/projects/a8d0cdd646d7462cb1f6a45b4040107f/users/f90be4fa47814fb38a40f79c2f494f9a/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.142791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.151348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 798/1608] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:50.152215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.153543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.174691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d0cdd646d7462cb1f6a45b4040107f, user_id=f90be4fa47814fb38a40f79c2f494f9a, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.180128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77c613-ddaf-4956-b1b2-3b9456c6de1f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.216645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758399b-1eda-4e61-9368-8fbe80e26628 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.220677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.221341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.221603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.221876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.232534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.234033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.245291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 799/1609] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:50.246019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.254979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5937ee-bf28-46b4-82ee-9be15dd6f18e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.259695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.260356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.260649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.260908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.275346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.282439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46279ca3-a267-49e6-a25a-2e9bbb532b17 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.289827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 800/1610] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:50.291136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.291493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.291727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.317332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 811/1611] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:50 2022] PUT /identity/v3/projects/a8d0cdd646d7462cb1f6a45b4040107f/users/f90be4fa47814fb38a40f79c2f494f9a/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:50.319989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.322585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.322585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.324497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] MFA Rules not processed for user `9221e818e45c4fb0987c9527cee3868e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:50.326307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.358087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] MFA Rules not processed for user `c2c50b74a2d34acd9656b40596682d13`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:50.360879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.419021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 801/1612] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:50.419926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.479441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 812/1613] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:50.481799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.505487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c6568-b2ca-4db9-bd82-95dfe855c74b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.510086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.510711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.511232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.517735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.541602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:50.577743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624931ae-e25a-4708-bd5b-d6be8d792389 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.587542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.588484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.588759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.590081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.605836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.607908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.609524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a80f90-23be-4614-a916-6de6b824f313 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.609836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.623581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 813/1614] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:50.625258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d817-196c-492a-a12e-94751468448f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.625848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d817-196c-492a-a12e-94751468448f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.625848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0d817-196c-492a-a12e-94751468448f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.631318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 802/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:50.633574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.648895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6076f-c425-4ccf-9a7f-b4bee0296b5b service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.655708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.656837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.657290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.657734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.658593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.661110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c9f0d817-196c-492a-a12e-94751468448f None None] MFA Rules not processed for user `f90be4fa47814fb38a40f79c2f494f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:50.662165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.662936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c9f0d817-196c-492a-a12e-94751468448f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:50.701465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:50.750856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 814/1616] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:50.752080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0d817-196c-492a-a12e-94751468448f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.766540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0d817-196c-492a-a12e-94751468448f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:50.781485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:50.781485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:50.781485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:50.781485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:50.782967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:50.782967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:50.910529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 803/1617] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:50.920257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:50.920414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 815/1618] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:50.920414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.018570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22366aec-37a1-41d7-b0f4-e986121a6b99 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.024741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.024964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7041b2a-5ab3-4367-8c52-5381d84d6722 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.025366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.025644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.025915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.026564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.029181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.031630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.032549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.032895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.033227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.033944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.036161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.222857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 816/1619] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/users => generated 297 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:51.223647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.234460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 804/1620] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:32:50 2022] POST /identity/v3/projects => generated 363 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:51.235674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.245528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95daca5f-f2b7-46ea-838d-217d30d84a64 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.338544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.339601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.340086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.340561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.342007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.344396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.372374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aaf1e7-75e7-4893-bb8d-32b6a6fd8491 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.378460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.378693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.378979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.379644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.380927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.382806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.398204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 805/1621] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:51.398204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.408900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1256ac-2fc1-4254-982e-243a9aa10273 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.414948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.415599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.415875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.416144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.416778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.418766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.466534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 817/1622] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:51 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:51.466534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.482760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697aca47-e9ea-4c26-b5b0-f34661486500 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.488860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.489876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.490231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.490631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] PATH_INFO: `/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8/users/9451bfda150f4aff85562d84ad9edb63/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.510044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce1b421a25f4c2183ed7356ce4005a8, user_id=9451bfda150f4aff85562d84ad9edb63, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.512953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.583791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 806/1623] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:32:51 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:32:51.584851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.592069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:51.603409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 818/1624] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:51 2022] PUT /identity/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8/users/9451bfda150f4aff85562d84ad9edb63/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:51.604584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.609714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:51.637783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4654a7a-15ef-4eb7-ae5d-1196884712a9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.757618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.758262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.758539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.758810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.759507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.761984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.771401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7de3f-ff91-4df5-aa87-607cc1d5a7a2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.781008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 807/1625] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:51.781332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.782476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.782695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.783258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.784237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.786209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.796030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95eab1-8b74-481a-b64d-cbfe5967728e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.803725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.804453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.804933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.805320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.805980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.807489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.813080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.825830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 808/1626] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:51.826766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.853219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 819/1627] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:51.853219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:51.853600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7dd9f-ee36-4677-a7a9-1a51cdcc4e27 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.861780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.863287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.866845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e54a0f-bfdd-422f-a9e5-595bd909d3c6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:51.867386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.868196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] PATH_INFO: `/v3/projects/dcafe1965b134d64bd15d3aec03df12d/users/5ca1cfcd6e074ba28ba858a4e626d592/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.882552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:51.882552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:51.882552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:51.882552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] PATH_INFO: `/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8/users/9451bfda150f4aff85562d84ad9edb63/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:51.889255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce1b421a25f4c2183ed7356ce4005a8, user_id=9451bfda150f4aff85562d84ad9edb63, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.892469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.908596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcafe1965b134d64bd15d3aec03df12d, user_id=5ca1cfcd6e074ba28ba858a4e626d592, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:51.912209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af315ccd-7216-4db4-9aef-ded9e7c90637 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:51.921783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 820/1628] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:51 2022] PUT /identity/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8/users/9451bfda150f4aff85562d84ad9edb63/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:51.922823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.058873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cbb84-ef93-4871-b33e-9923340644dd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.070552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.070552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.070552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.070552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] PATH_INFO: `/v3/projects/21f7872842f84dc7b317b3af1fcd9d66/users/2681b166e47f4d83bb1b58e857afa850/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.093902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f7872842f84dc7b317b3af1fcd9d66, user_id=2681b166e47f4d83bb1b58e857afa850, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.098311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.122782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 809/1629] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:51 2022] PUT /identity/v3/projects/dcafe1965b134d64bd15d3aec03df12d/users/5ca1cfcd6e074ba28ba858a4e626d592/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:52.124573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.125075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.125765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.164550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] MFA Rules not processed for user `9451bfda150f4aff85562d84ad9edb63`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:52.166335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:52.203561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 821/1630] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:51 2022] PUT /identity/v3/projects/21f7872842f84dc7b317b3af1fcd9d66/users/2681b166e47f4d83bb1b58e857afa850/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:52.204710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.308790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 810/1631] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:52.310279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.375729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75350cc-46d5-4e12-a5e8-46828fa4b2dc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.386585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.387538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.392114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.392114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.392114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.394279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.400385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f397d5-493f-4dc1-bb7f-8b0bfca2a73e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.406548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.408034 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.408448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.408816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.409581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.413168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.420026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 822/1632] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:52.421069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.425641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 811/1633] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:32:52.426597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.445257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd702f4-6fa0-4f78-9616-c101462c0346 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.451759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.452438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.452713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.453028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] PATH_INFO: `/v3/projects/dcafe1965b134d64bd15d3aec03df12d/users/5ca1cfcd6e074ba28ba858a4e626d592/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.455514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcafe1965b134d64bd15d3aec03df12d, user_id=5ca1cfcd6e074ba28ba858a4e626d592, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.459825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.476421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 812/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:52 2022] PUT /identity/v3/projects/dcafe1965b134d64bd15d3aec03df12d/users/5ca1cfcd6e074ba28ba858a4e626d592/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:52.477814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.538743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9049afa-dcf1-4f32-9d02-b2c3c7840f2c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.569677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12faa97-1fd6-41aa-a695-01eca3123527 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.569677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.570279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.570558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.570829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] PATH_INFO: `/v3/projects/21f7872842f84dc7b317b3af1fcd9d66/users/2681b166e47f4d83bb1b58e857afa850/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.573361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f7872842f84dc7b317b3af1fcd9d66, user_id=2681b166e47f4d83bb1b58e857afa850, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.576821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe9a11-3707-469d-956a-2253dbb6a1df admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.589905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 813/1635] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:32:52 2022] PUT /identity/v3/projects/21f7872842f84dc7b317b3af1fcd9d66/users/2681b166e47f4d83bb1b58e857afa850/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:32:52.591756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.592115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.592449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.627402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.628623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.629237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.629531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.637193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] MFA Rules not processed for user `5ca1cfcd6e074ba28ba858a4e626d592`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:52.639127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:52.751902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:52.756655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30b193-d7d0-4bff-babf-d982814748e6 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:52.779012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 823/1636] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:52.781674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.782487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.784847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.824609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] MFA Rules not processed for user `2681b166e47f4d83bb1b58e857afa850`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:32:52.825880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:32:52.831073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 814/1637] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:52.836489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.836768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:52.938743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b656f5-f942-44af-85e9-e6c392de3c22 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:52.946164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:52.952558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:52.952558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:52.952558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:52.955646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 824/1638] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:32:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:32:52.985665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:52.999373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156fcc2a-41b3-4dc2-8525-4bc626c5fde1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:53.004325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:53.006264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:53.007462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:53.008632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:53.025088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:53.028544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:53.042431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 825/1639] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:32:53.045532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:53.047831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:53.058816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 815/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:53.117999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:54.167880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:54.183618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6d5ae-d67d-4401-b9a1-dce36b913041 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:54.191296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:54.191986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:54.192320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:54.192541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:54.280567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:54.282350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ea842-51a4-422c-8153-ab669eb7d7e1 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:54.294494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 826/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:54.790365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:32:55.020582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:32:55.030180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d655da4-b8a0-40ce-ac64-0b6f62ea5528 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:32:55.034473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:32:55.035451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:32:55.035867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:32:55.036290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:32:55.149697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:32:55.151950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1ba91-0eb0-4d0e-947d-4a6dd7a90f66 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:32:55.172693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 816/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:32:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:32:55.270238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:03.226147 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7211-9da3-45dc-9b1c-dd73083871bc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:03.226935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7211-9da3-45dc-9b1c-dd73083871bc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:03.226935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f7211-9da3-45dc-9b1c-dd73083871bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:03.279144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1d1f7211-9da3-45dc-9b1c-dd73083871bc None None] MFA Rules not processed for user `8937e9f80f1540e389ee283971624741`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:03.280527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1d1f7211-9da3-45dc-9b1c-dd73083871bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:03.392726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 827/1643] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:03.407207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cba2c0-44c1-485f-8818-130a07911e24 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:03.407207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cba2c0-44c1-485f-8818-130a07911e24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:03.407207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cba2c0-44c1-485f-8818-130a07911e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:03.445735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-81cba2c0-44c1-485f-8818-130a07911e24 None None] MFA Rules not processed for user `8937e9f80f1540e389ee283971624741`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:03.446795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-81cba2c0-44c1-485f-8818-130a07911e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:03.460204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 817/1644] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:03.504898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb351c-d5bb-4533-98a8-7ec0ccc54cce None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:03.505881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb351c-d5bb-4533-98a8-7ec0ccc54cce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:03.506166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb351c-d5bb-4533-98a8-7ec0ccc54cce None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:03.510084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 828/1645] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 16:33:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:03.541463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65d117-dfa0-4b9b-9836-3ffac868b4e4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:03.542368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65d117-dfa0-4b9b-9836-3ffac868b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:03.542872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65d117-dfa0-4b9b-9836-3ffac868b4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:03.622671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fc65d117-dfa0-4b9b-9836-3ffac868b4e4 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:03.625141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc65d117-dfa0-4b9b-9836-3ffac868b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:03.751612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 818/1646] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 16:33:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:03.753576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb351c-d5bb-4533-98a8-7ec0ccc54cce None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:03.776223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb351c-d5bb-4533-98a8-7ec0ccc54cce None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:03.789971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:03.790628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:03.791120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:03.791488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:03.801843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:03.804401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122792e3-44d6-49e6-9f51-13f4a86641bc service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:03.829677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 829/1647] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:04.362655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:04.363173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:04.363788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:04.366412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 819/1648] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 16:33:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:04.373484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497e19-9d89-436e-84fd-c818cf7723b8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:04.375122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497e19-9d89-436e-84fd-c818cf7723b8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:04.376254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497e19-9d89-436e-84fd-c818cf7723b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:04.433938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-40497e19-9d89-436e-84fd-c818cf7723b8 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:04.444153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-40497e19-9d89-436e-84fd-c818cf7723b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:04.478873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 830/1649] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:04.496283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:04.502306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:04.556964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:04.723703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8992f-b68f-46ec-bf53-eea607ce6815 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:04.730538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:04.731565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:04.732745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:04.733038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:04.770217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:04.774049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415d583-aee7-40aa-8c6d-81ceec48d1e3 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:04.811368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 820/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:06.332572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a86f5-b2f5-4875-b814-83a0b8c7963a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:06.332572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a86f5-b2f5-4875-b814-83a0b8c7963a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:06.332572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a86f5-b2f5-4875-b814-83a0b8c7963a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:06.332572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 831/1651] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 16:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:06.340796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:06.341254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:06.341508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:06.468054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:06.471287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:06.629823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220694-b5f9-4cc2-a774-20351335af8e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:06.630469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220694-b5f9-4cc2-a774-20351335af8e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:06.631112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5220694-b5f9-4cc2-a774-20351335af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:06.728145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 821/1652] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:06.745294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f5220694-b5f9-4cc2-a774-20351335af8e None None] MFA Rules not processed for user `8a45ee95cf5947c49584ff403facaff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:06.747144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5220694-b5f9-4cc2-a774-20351335af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:06.777134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:06.790372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f5050-5d4a-4645-aed9-047170ac0f73 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:06.812418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:06.819629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:06.819933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:06.820214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:06.851409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:06.855562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882a2fea-9083-4f9b-a0e0-e84c2f1aeae7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:06.880594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 822/1653] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:06.885678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:06.993828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 832/1654] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:07.048628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7d300-d134-4638-bd7f-669bff4a661a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:07.048955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7d300-d134-4638-bd7f-669bff4a661a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:07.049549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7d300-d134-4638-bd7f-669bff4a661a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:07.050970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 823/1655] 10.222.0.16 () {54 vars in 962 bytes} [Thu Sep 1 16:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:07.103541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:07.104132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:07.104684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:07.150405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:07.152548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:07.183412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 833/1656] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Sep 1 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:07.202296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7d300-d134-4638-bd7f-669bff4a661a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:07.234766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:07.259941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7d300-d134-4638-bd7f-669bff4a661a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:07.268456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd484254-f869-4a0d-ad16-160641e8208d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:07.278641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:07.280513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:07.281348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:07.281875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:07.282685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:07.284220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:07.286155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:07.287557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:07.316523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:07.317434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:07.320660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731f386-0c18-4120-a5e0-b911733caecd service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:07.321495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:07.359398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 834/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:07.360223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 824/1658] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:08.277967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:08.388724 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dafa-e84f-465a-8902-ad7a807db6c0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:08.393374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dafa-e84f-465a-8902-ad7a807db6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:08.393374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8dafa-e84f-465a-8902-ad7a807db6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:08.446105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2dc8dafa-e84f-465a-8902-ad7a807db6c0 None None] MFA Rules not processed for user `8937e9f80f1540e389ee283971624741`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:08.450854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2dc8dafa-e84f-465a-8902-ad7a807db6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:08.470040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 825/1659] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:08.506861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593baa22-0b24-4580-a216-f331967a6aa8 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:08.536119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:08.541137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:08.541429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:08.541707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:08.549594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:08.593088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:08.596479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753c99c-f79d-4df9-8bbb-60a8d04c02f9 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:08.624523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 835/1660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:08.632319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:08.632319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:08.632781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:08.634781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 826/1661] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 16:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:08.640570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d0a6-d327-4ad4-9e31-e1664d8a9f83 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:08.640966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d0a6-d327-4ad4-9e31-e1664d8a9f83 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:08.641340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87d0a6-d327-4ad4-9e31-e1664d8a9f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:08.704531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3a87d0a6-d327-4ad4-9e31-e1664d8a9f83 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:08.706564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3a87d0a6-d327-4ad4-9e31-e1664d8a9f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:08.737326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 836/1662] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:08.751478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:08.753758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:08.792818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ef8df-eaea-4022-9875-195d1d87a249 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:08.820758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:08.821401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:08.821867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:08.822141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:08.856874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:08.858365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b55fa3-e8db-4f87-a07e-959b22fb7c96 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:08.883891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 827/1663] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:09.314137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fec2-0c82-403b-aec2-d8612e4a0657 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:09.314547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fec2-0c82-403b-aec2-d8612e4a0657 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:09.314930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6fec2-0c82-403b-aec2-d8612e4a0657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:09.438270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-09a6fec2-0c82-403b-aec2-d8612e4a0657 None None] MFA Rules not processed for user `662f85837db7411e8695d971413fc5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:09.439485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-09a6fec2-0c82-403b-aec2-d8612e4a0657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:09.454891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19876c-4484-4865-bc36-0ddd77ea3fd4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:09.455551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19876c-4484-4865-bc36-0ddd77ea3fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:09.456247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19876c-4484-4865-bc36-0ddd77ea3fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:09.583737 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bb19876c-4484-4865-bc36-0ddd77ea3fd4 None None] MFA Rules not processed for user `ab4df1724851433889a519583a5333b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:09.587963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bb19876c-4484-4865-bc36-0ddd77ea3fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:09.657890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 837/1664] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:09.670571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:09.670933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:09.671379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:09.722814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] MFA Rules not processed for user `662f85837db7411e8695d971413fc5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:09.729499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:09.766359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 838/1665] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:09.777314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:09.796470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:09.823347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200ede8-bab2-4ef4-b43d-1f9865e2ac94 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:09.839803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:09.840946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:09.841675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:09.845709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:09.894806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:09.896530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 828/1666] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:09.898917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc2fc6-5f9c-47e4-be8e-0d6623727eb9 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:09.948826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 839/1667] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:10.158360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd31dd-d147-40a7-af0b-7f1754a5412f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:10.158360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd31dd-d147-40a7-af0b-7f1754a5412f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:10.158360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd31dd-d147-40a7-af0b-7f1754a5412f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:10.165561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 829/1668] 10.222.0.16 () {54 vars in 912 bytes} [Thu Sep 1 16:33:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:10.171985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1b6e6-3b27-4855-8185-5d5ad2cda0b5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:10.172705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1b6e6-3b27-4855-8185-5d5ad2cda0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:10.173238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1b6e6-3b27-4855-8185-5d5ad2cda0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:10.224779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bed1b6e6-3b27-4855-8185-5d5ad2cda0b5 None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:10.226144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bed1b6e6-3b27-4855-8185-5d5ad2cda0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:10.245295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 840/1669] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:10.268372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd31dd-d147-40a7-af0b-7f1754a5412f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:10.315858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd31dd-d147-40a7-af0b-7f1754a5412f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:10.330335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:10.336272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:10.337130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:10.338800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:10.385254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:10.388349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:10.414685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 830/1670] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:11.038070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:11.048574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:11.048574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:11.052937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:11.137373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] MFA Rules not processed for user `21e6ff7908e84a5994a51be0cfc82414`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:11.137373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:11.317103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 841/1671] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:11.436774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:11.470542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e8a43-1316-4b57-97c1-7f48efd385b4 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:11.478352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:11.479281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:11.479760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:11.480206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:11.567874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:11.575180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6739124-08fc-4246-bf1b-fc0376701eb8 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:11.600090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 831/1672] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:11.749636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:11.775019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24089b-694e-43ed-9988-4cdd94a31820 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:11.808551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:11.809228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:11.809508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:11.809854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:11.836065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:11.837869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707cc7e9-b94b-4720-8520-1c265f6ae768 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:11.878019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 842/1673] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:14.225634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151c986-f849-41f2-8cdf-1f8a2b6b2107 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:14.226129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151c986-f849-41f2-8cdf-1f8a2b6b2107 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:14.226321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151c986-f849-41f2-8cdf-1f8a2b6b2107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:14.270792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a151c986-f849-41f2-8cdf-1f8a2b6b2107 None None] MFA Rules not processed for user `8a45ee95cf5947c49584ff403facaff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:14.272984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a151c986-f849-41f2-8cdf-1f8a2b6b2107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:14.283581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 832/1674] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:17.248056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd418-981f-4ef6-bf16-613d84ee96c8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.249130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd418-981f-4ef6-bf16-613d84ee96c8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.249578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fd418-981f-4ef6-bf16-613d84ee96c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.281300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4d3fd418-981f-4ef6-bf16-613d84ee96c8 None None] MFA Rules not processed for user `ab4df1724851433889a519583a5333b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:17.283025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4d3fd418-981f-4ef6-bf16-613d84ee96c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.293158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 843/1675] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:17.497160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:17.497738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:17.497738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:17.535550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] MFA Rules not processed for user `21e6ff7908e84a5994a51be0cfc82414`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:17.538070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:17.552608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 833/1676] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:22.026760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:22.031140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:22.031140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:22.110439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:22.112125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:22.385741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 844/1677] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:33:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:23.002596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:23.004526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:23.243101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad329a6-8b8f-459f-8490-2e3c1b3b3a1a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:23.258705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:23.259604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:23.260011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:23.261322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:23.275935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:23.280139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a44bfc-8074-425b-83ff-3d92b8f32df3 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:23.300531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 834/1678] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:33:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:33:28.975255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:29.110827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:29.155463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41498b0-b35d-49d7-815f-4fb2ddeb60c3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.167183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.168104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.168949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.169572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] PATH_INFO: `/v3/users/9221e818e45c4fb0987c9527cee3868e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.171046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9221e818e45c4fb0987c9527cee3868e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.173278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:29.462244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9221e818e45c4fb0987c9527cee3868e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.547202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9221e818e45c4fb0987c9527cee3868e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.548307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9221e818e45c4fb0987c9527cee3868e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.566479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9221e818e45c4fb0987c9527cee3868e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.568737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9221e818e45c4fb0987c9527cee3868e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:29.606834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 845/1679] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:28 2022] DELETE /identity/v3/users/9221e818e45c4fb0987c9527cee3868e => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:29.608504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.609090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.609090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.645245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:29.651711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:29.748295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 835/1680] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:29.765861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:29.768605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:29.924584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad647be-fdc7-4fc8-975c-ab09e50d6cf3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:29.928442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:29.929321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:29.929773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:29.930198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:29.945220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:29.947224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:29.960314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 846/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:30.896182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:30.904877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:30.919693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f439996f-903d-4003-b64c-fad83c314d8c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:30.926327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:30.927867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:30.933482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:30.933482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] PATH_INFO: `/v3/projects/5133df6dc2af4bd0bf409bc7a6678ebc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:30.933482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5133df6dc2af4bd0bf409bc7a6678ebc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:30.933482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:30.937347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:31.138645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5133df6dc2af4bd0bf409bc7a6678ebc', 'request_id': 'req-15105688-7c2d-4bec-9fef-f63da171aeb0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.149552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5133df6dc2af4bd0bf409bc7a6678ebc', 'request_id': 'req-15105688-7c2d-4bec-9fef-f63da171aeb0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.265231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5133df6dc2af4bd0bf409bc7a6678ebc', 'request_id': 'req-15105688-7c2d-4bec-9fef-f63da171aeb0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.282210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] The token cache is being invalidate because project 5133df6dc2af4bd0bf409bc7a6678ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:31.282753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.286020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.289195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 836/1682] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:30 2022] DELETE /identity/v3/projects/5133df6dc2af4bd0bf409bc7a6678ebc => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:31.293268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:31.304411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ef4a8-9fbc-4eda-908d-26270651bef2 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:31.516334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:31.516334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.516334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.516334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] PATH_INFO: `/v3/users/9451bfda150f4aff85562d84ad9edb63` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.516903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9451bfda150f4aff85562d84ad9edb63)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:31.518854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:31.545503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:31.814456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9451bfda150f4aff85562d84ad9edb63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.893622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:31.895105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:31.901283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9451bfda150f4aff85562d84ad9edb63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.901948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9451bfda150f4aff85562d84ad9edb63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.909943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105688-7c2d-4bec-9fef-f63da171aeb0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:31.915745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:31.916120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9451bfda150f4aff85562d84ad9edb63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.916534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:31.916824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bbc501bd-f9ac-4413-9ebe-fd1c2d31dc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9451bfda150f4aff85562d84ad9edb63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:31.916950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:31.917156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] PATH_INFO: `/v3/users/f90be4fa47814fb38a40f79c2f494f9a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:31.918293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f90be4fa47814fb38a40f79c2f494f9a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:31.919370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:31.919370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Sep 01 16:33:31.919370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: Sep 01 16:33:31.920599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:31.929461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 847/1683] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:31 2022] DELETE /identity/v3/users/9451bfda150f4aff85562d84ad9edb63 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:32.175920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f90be4fa47814fb38a40f79c2f494f9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.176625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f90be4fa47814fb38a40f79c2f494f9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.283648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f90be4fa47814fb38a40f79c2f494f9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.295787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f90be4fa47814fb38a40f79c2f494f9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.317839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f90be4fa47814fb38a40f79c2f494f9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.327681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 837/1684] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:31 2022] DELETE /identity/v3/users/f90be4fa47814fb38a40f79c2f494f9a => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:32.332364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.335050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.335050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.353307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:32.354519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.414937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 848/1685] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:32.427114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.427114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.430377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.435884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.450173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbfdfdc-dcbd-406f-bbb1-b52d16ea5b29 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.456996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.457648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.458542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.458820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] PATH_INFO: `/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.460257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce1b421a25f4c2183ed7356ce4005a8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.461440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 01 16:33:32.461440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Sep 01 16:33:32.461440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: Sep 01 16:33:32.462655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.491888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8debc3-0335-424c-8eaa-1190fd20b902 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.496301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.497059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.497409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.497755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.507606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.509461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f87e2-10ec-4ae5-ae02-ab1f7fe8fea3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.521389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 838/1686] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:32.615328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce1b421a25f4c2183ed7356ce4005a8', 'request_id': 'req-31028a71-d6e5-4845-b9e4-a5db0c422b4b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.630869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce1b421a25f4c2183ed7356ce4005a8', 'request_id': 'req-31028a71-d6e5-4845-b9e4-a5db0c422b4b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.729243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce1b421a25f4c2183ed7356ce4005a8', 'request_id': 'req-31028a71-d6e5-4845-b9e4-a5db0c422b4b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.741588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] The token cache is being invalidate because project 6ce1b421a25f4c2183ed7356ce4005a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:32.741875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.742610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-31028a71-d6e5-4845-b9e4-a5db0c422b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:32.744171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 849/1687] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:32 2022] DELETE /identity/v3/projects/6ce1b421a25f4c2183ed7356ce4005a8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:32.749156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.749448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.749726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.764990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:32.771801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:32.772958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.865145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 839/1688] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:32.873478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.873775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.874076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.913753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:32.916232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:32.929954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 850/1689] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:32.934451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.949598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297467ad-2e0a-4cd7-a3d0-cdd59cb89bfb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.954914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.955730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.956088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.956428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.957157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:32.957560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.960245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:32.972669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 840/1690] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:32.977480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:32.983333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:32.987248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18441118-b665-4edc-a023-51f4b5a9cc26 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:32.992218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:32.992828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:32.993102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:32.993369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:32.994026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:32.996007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.068904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.070808 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.086819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd6e9c-7f22-44a4-a8b8-e8b07c26ee47 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.092654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.093361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.093600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.093866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] PATH_INFO: `/v3/projects/a8d0cdd646d7462cb1f6a45b4040107f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.095028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d0cdd646d7462cb1f6a45b4040107f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.099598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.132113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 851/1691] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:32 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:33.136279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.207312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01d064-ce51-4b8a-8301-621852e55be7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.213341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.213957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.214234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.214505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.215166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.217090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.240794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d0cdd646d7462cb1f6a45b4040107f', 'request_id': 'req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.250658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d0cdd646d7462cb1f6a45b4040107f', 'request_id': 'req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.313226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 852/1692] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:33.317187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.327728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c1bc0b-06eb-4f93-ad80-9871d8c5d1a4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.382268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d0cdd646d7462cb1f6a45b4040107f', 'request_id': 'req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.385753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.391131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.391131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.391131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.391131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.391131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.395518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] The token cache is being invalidate because project a8d0cdd646d7462cb1f6a45b4040107f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:33.395819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.396556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.398091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 841/1693] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:33 2022] DELETE /identity/v3/projects/a8d0cdd646d7462cb1f6a45b4040107f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:33.400781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 853/1694] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:33.405177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.405528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.406946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.407550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.419908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c0a59-de14-4fa7-8a72-b03fbed8fe22 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.420623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d8c5b3-51df-4d31-85ab-f3570fd0b933 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.424625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.425293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.425549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.425663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.425828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.426168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.426446 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.426539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.427342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] PATH_INFO: `/v3/users/2681b166e47f4d83bb1b58e857afa850` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.428493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2681b166e47f4d83bb1b58e857afa850)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.429879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.430539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.442559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 854/1695] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:33.447428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.457283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7be8c1-a161-4b93-8481-a8ba60f27e6b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.462832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.463548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.463823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.464097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] PATH_INFO: `/v3/projects/43245dd76e294fc1b8f2a3b393c54f11/users/cb847ab3431546d4bb89842150341af8/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.479492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=43245dd76e294fc1b8f2a3b393c54f11, user_id=cb847ab3431546d4bb89842150341af8, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.482230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8e2c1-4af3-4663-b358-8e9f1f98ddfb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:33.562014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 855/1696] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:33 2022] PUT /identity/v3/projects/43245dd76e294fc1b8f2a3b393c54f11/users/cb847ab3431546d4bb89842150341af8/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:33.573666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.574005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.574346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.630983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] MFA Rules not processed for user `cb847ab3431546d4bb89842150341af8`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:33.634278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.683667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2681b166e47f4d83bb1b58e857afa850'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.684887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2681b166e47f4d83bb1b58e857afa850'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.744096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 856/1697] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:33.745163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:33.747125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:33.778671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2681b166e47f4d83bb1b58e857afa850'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.788029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2681b166e47f4d83bb1b58e857afa850'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.803744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2681b166e47f4d83bb1b58e857afa850'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:33.812315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 842/1698] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:33 2022] DELETE /identity/v3/users/2681b166e47f4d83bb1b58e857afa850 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:33.835632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631a477-60ea-4894-b56b-3190cfab2c00 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:33.907586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:33.908216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:33.908570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:33.908852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:33.909646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:33.911919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.036129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 857/1699] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:33 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:34.038016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.057395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433267e2-6672-4811-a48a-7bbe2a1f44e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.170613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.171473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.171895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.172234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.173087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.175582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.282340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:34.323619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 843/1700] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/users => generated 294 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:34.327461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.346612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d04450-3aa2-432c-9432-05d38c8eb416 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.374697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.376603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.379482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.380414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.381600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.383563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.399987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 858/1701] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:34.403989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.418167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b4347-1dfd-495e-a200-988b09d4b50c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.427239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.428347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.428830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.429364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.430254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.432643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.447481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 844/1702] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:34.452267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.467306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.468074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6fe5d-7474-4afa-a407-505ce2d30a6a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.470099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.486114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.487874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.496145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.496145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] PATH_INFO: `/v3/projects/5a874f6a61134c138d582b0a84358df0/users/4a9a4a38864540bd9ea690123b5d58c9/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.511554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847efecf-9fd1-449a-b24a-4c224cafd135 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.522964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.522964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.522964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.523389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] PATH_INFO: `/v3/projects/21f7872842f84dc7b317b3af1fcd9d66` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.524833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f7872842f84dc7b317b3af1fcd9d66)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.527188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.573145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:34.577687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a874f6a61134c138d582b0a84358df0, user_id=4a9a4a38864540bd9ea690123b5d58c9, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:34.582838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d7b4d-efe1-4917-8ee7-881afce5bcbe admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:34.700150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 859/1703] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:34 2022] PUT /identity/v3/projects/5a874f6a61134c138d582b0a84358df0/users/4a9a4a38864540bd9ea690123b5d58c9/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:34.702163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.702453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.702725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.711280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f7872842f84dc7b317b3af1fcd9d66', 'request_id': 'req-abedb2ba-8240-40a4-b048-9796ca41eea7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.721384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f7872842f84dc7b317b3af1fcd9d66', 'request_id': 'req-abedb2ba-8240-40a4-b048-9796ca41eea7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.771793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] MFA Rules not processed for user `4a9a4a38864540bd9ea690123b5d58c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:34.773606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.818600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f7872842f84dc7b317b3af1fcd9d66', 'request_id': 'req-abedb2ba-8240-40a4-b048-9796ca41eea7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.847663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] The token cache is being invalidate because project 21f7872842f84dc7b317b3af1fcd9d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:34.848750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.850855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-abedb2ba-8240-40a4-b048-9796ca41eea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:34.854597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 845/1704] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:34 2022] DELETE /identity/v3/projects/21f7872842f84dc7b317b3af1fcd9d66 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:34.861470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c10dfd-aab3-46f7-99cc-7bf75b5d61a9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.861961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c10dfd-aab3-46f7-99cc-7bf75b5d61a9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.862337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c10dfd-aab3-46f7-99cc-7bf75b5d61a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.868472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 860/1705] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:34.873161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:34.932643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-94c10dfd-aab3-46f7-99cc-7bf75b5d61a9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:34.933949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94c10dfd-aab3-46f7-99cc-7bf75b5d61a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:34.981331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50634b14-2da4-4324-a9c1-1f6540c7160b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:34.992106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:34.995003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:34.995315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:34.995577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:34.997843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.000701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.093001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 846/1706] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:35.117046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.117336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.117630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.143129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:35.151216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.168494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 861/1707] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:34 2022] POST /identity/v3/projects => generated 385 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:35.168598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 847/1708] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:35.168598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.168837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.170630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.177932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a020e-e6b5-4ed6-8da5-7e98e1ae6072 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.242209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:35.256011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.256011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.256011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.256011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.256593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:35.256593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.256593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.258132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962040b-0acc-453a-8ab7-fe68f1f58086 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.273611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 848/1709] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:35.274600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.275196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.275301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.275640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.275857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.276646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.278521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.284362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:35.297693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd735b-3ed6-4493-a2ce-0bf40530bb38 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.303306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.303947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.304213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.304484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.305623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.308140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.469091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 862/1710] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/users => generated 299 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:35.470109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.481841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5660760-3827-426c-af38-ec2f509fe523 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.486682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.487373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.487654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.487923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.488572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.491020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.506524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 863/1711] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:35.508172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.518743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695db9-f18d-4951-a23b-76d72a873174 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.524808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.526502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.526789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.527070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] PATH_INFO: `/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00/users/d34006d2f81342fc8240c98b5b1d565c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.539377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 849/1712] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/projects => generated 369 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:35.540503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.545094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8e7b3078a741f7bb8dc963783a4e00, user_id=d34006d2f81342fc8240c98b5b1d565c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.548357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.598363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:35.613180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb052f2-197a-43d6-a8a4-5623fa3fa3ef admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.626456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.627110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.627442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.627711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.628355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.631521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.654929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 864/1713] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:35 2022] PUT /identity/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00/users/d34006d2f81342fc8240c98b5b1d565c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:35.655872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.727201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c858c4-d57b-47ef-9690-90a8cf46a7b1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.733769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.734523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.734905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.735405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.736314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 850/1714] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:35.736431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.738226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.740991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.756519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 865/1715] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:35.757839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.761665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f53932-0e59-46a2-9b82-b79cee93ede9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.768566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.768770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a6e824-cbe9-4932-965d-e352d7fec4af admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.769202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.769611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.769927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.771223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.773370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.774326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.774995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.775306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.779601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] PATH_INFO: `/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00/users/d34006d2f81342fc8240c98b5b1d565c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.779601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8e7b3078a741f7bb8dc963783a4e00, user_id=d34006d2f81342fc8240c98b5b1d565c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.780437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d073ff5-830d-4f08-9044-2bfb7744e121 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.787595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 851/1716] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:35.791576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.795674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 866/1717] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:35 2022] PUT /identity/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00/users/d34006d2f81342fc8240c98b5b1d565c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:35.807374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf8439-fade-4c12-99bc-83a08bdff617 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.808735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.809034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.809316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.849030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] MFA Rules not processed for user `d34006d2f81342fc8240c98b5b1d565c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:35.854489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.893733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.898527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:35.914793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 852/1718] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:33:35.925127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.944871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42553d2-150a-4643-99ec-89bd873a8ab2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:35.951234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:35.953198 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:35.955323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 867/1719] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:35.955437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:35.955437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] PATH_INFO: `/v3/projects/5a0d03f36dbd4ecf9e5168ceb842f5a3/users/78ec8af92c024456bc2779f218c32150/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:35.956732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:35.966324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:35.978929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:35.991202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a0d03f36dbd4ecf9e5168ceb842f5a3, user_id=78ec8af92c024456bc2779f218c32150, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:35.994887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:36.054407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db65c55-e772-46b9-b618-c0e0f59915df None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:36.058257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:36.058885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.059188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.059468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.071873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:36.073334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfdb88-08ce-4c7e-9be9-10ff101e61eb service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:36.078763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:36.086644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 868/1720] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:36.143583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 853/1721] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:35 2022] PUT /identity/v3/projects/5a0d03f36dbd4ecf9e5168ceb842f5a3/users/78ec8af92c024456bc2779f218c32150/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:36.150946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.151293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.151573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.186268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] MFA Rules not processed for user `78ec8af92c024456bc2779f218c32150`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:36.187734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:36.286040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 869/1722] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:36.290316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:36.394571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dd3ce9-4ef0-4a98-8eee-098708bd633f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:36.404153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:36.404848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.405217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.405715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.406424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:36.408466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:36.554779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 854/1723] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:36 2022] POST /identity/v3/projects => generated 367 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:36.560464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:36.574208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb62c8-335d-4650-a04e-e6d7c507103f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:36.655707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:36.656290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.656647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.656951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.657673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:36.665701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:36.877195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 870/1724] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:36 2022] POST /identity/v3/users => generated 290 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:36.885002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:36.895829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d7019-c832-415a-9fdc-c0fd8362f12d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:36.901151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:36.901633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.901908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.902197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.902846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:36.904545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:36.921532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 855/1725] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:36.930374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:36.941871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0e4ac-845f-4766-b897-5eca25e0763f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:36.947517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:36.948191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:36.948470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:36.948742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] PATH_INFO: `/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469/users/b91a5e75391843c9a26473512e373912/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:36.970069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e9d99ab5ca4034ace4e71bcbc61469, user_id=b91a5e75391843c9a26473512e373912, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:36.972916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:37.122874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 871/1726] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:36 2022] PUT /identity/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469/users/b91a5e75391843c9a26473512e373912/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:37.127453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:37.139460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160b7f6-9ff4-4225-b164-30c6675acede admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:37.206768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:37.207583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:37.207759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:37.207991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:37.208635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:37.210249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15a928-524d-43ff-9c40-5629dd31608f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:37.221247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 856/1727] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:37.225881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:37.243177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6253ca-40ab-4f9d-bc52-b75727cca3a6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:37.249749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:37.251893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:37.251893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:37.251893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] PATH_INFO: `/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469/users/b91a5e75391843c9a26473512e373912/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:37.254103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e9d99ab5ca4034ace4e71bcbc61469, user_id=b91a5e75391843c9a26473512e373912, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:37.259483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:37.276103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 872/1728] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:37 2022] PUT /identity/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469/users/b91a5e75391843c9a26473512e373912/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:37.288729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:37.289058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:37.289373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:37.326427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] MFA Rules not processed for user `b91a5e75391843c9a26473512e373912`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:37.327800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:37.336067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:37.418782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 857/1729] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:37.446543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:37.450251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:37.547972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:37.559505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:37.580144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6aac9-319c-493e-a4c7-8219fbb8418f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:37.587050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:37.588077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:37.588715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:37.589160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:37.598085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433455f5-177a-4593-bf4c-0e835ad7c7eb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:37.611426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:37.612118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:37.612366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:37.613048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:37.617935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:37.624015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230e776d-9907-467e-865a-1f8a42acf134 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:37.629105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:37.641503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 873/1730] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:37.717403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:37.719085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:37.733725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 858/1731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:38.606758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:38.607321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:38.607321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:38.675293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:38.677593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:38.882373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 874/1732] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Sep 1 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:38.990778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:38.992404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:38.994262 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:38.996167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:39.013949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858d275-70a4-40f3-8b9d-2d68b2ed96d5 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:39.017121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:39.017436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:39.017789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:39.018134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:39.027980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:39.029824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:39.031588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ae87-90a4-480b-97ca-3b563ee234df None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:39.040314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:39.040526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 859/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:39.049126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:39.049376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:39.049376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:39.049376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:39.060015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:39.061462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f5b15c-5a80-49c6-82c1-7b784acda2f0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:39.077839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35734f03-1548-4953-8b5e-cfac0481b553 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:39.092942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:39.093681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:39.094084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:39.094371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:39.105795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 875/1734] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:39.108894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:39.110603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b5787-71c0-4487-aa9a-837ccb3d265d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:39.122266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 860/1735] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:41.256521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:41.257864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:41.258582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:41.260853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 876/1736] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:33:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:41.277607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb33d8-55bb-4240-a88c-7a9c9c00c916 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:41.281361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb33d8-55bb-4240-a88c-7a9c9c00c916 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:41.282641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb33d8-55bb-4240-a88c-7a9c9c00c916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:41.330781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-30fb33d8-55bb-4240-a88c-7a9c9c00c916 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:41.334137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30fb33d8-55bb-4240-a88c-7a9c9c00c916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:41.468167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 861/1737] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:41.523028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:41.525587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:41.550135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c6adf-2e45-4852-9e1a-74869be636db None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:41.563439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:41.581704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:41.581704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:41.581704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:41.581704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:41.584776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66af63a-1956-4380-8551-fc9e250e3f66 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:41.655447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 877/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:41.696572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:41.696930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:41.697641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:41.700103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 862/1739] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:33:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:41.748217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:41.748217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:41.748217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:41.786360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:41.788072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:41.809536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 878/1740] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:42.112736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:42.112736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:42.137412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f298404-b0d4-4723-9bde-8e457cc06f47 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:42.143698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:42.147916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:42.147916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:42.147916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:42.154218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:42.158552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:42.160487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30934ff-cdb6-4086-907e-61c5cf65892c service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:42.178788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 863/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:43.036655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:43.135963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2739c2-1a53-4c23-a2d7-d200d0cd7549 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:43.146093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:43.146093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:43.146093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:43.147454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] PATH_INFO: `/v3/users/78ec8af92c024456bc2779f218c32150` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:43.150154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ec8af92c024456bc2779f218c32150)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:43.153106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:43.455546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ec8af92c024456bc2779f218c32150'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:43.607639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ec8af92c024456bc2779f218c32150'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:43.608341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ec8af92c024456bc2779f218c32150'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:43.620645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:43.626766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ec8af92c024456bc2779f218c32150'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:43.627436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ec8af92c024456bc2779f218c32150'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:43.655362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 879/1742] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:43 2022] DELETE /identity/v3/users/78ec8af92c024456bc2779f218c32150 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:43.655587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:43.655587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:43.655587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:43.673997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:43.675481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:43.755944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 864/1743] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:43.766065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:43.774795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:43.849482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c73c-fbd8-4c8e-bf7e-5b36510b604f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:43.854604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:43.855286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:43.855571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:43.855907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:43.866315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:43.867923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:43.878954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 880/1744] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:44.559565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:44.564623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:44.574975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36bcb00-169f-4dc8-b86a-377a2197cece None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:44.580422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:44.581059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:44.581337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:44.581596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] PATH_INFO: `/v3/projects/5a0d03f36dbd4ecf9e5168ceb842f5a3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:44.585908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a0d03f36dbd4ecf9e5168ceb842f5a3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:44.587936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:44.731067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a0d03f36dbd4ecf9e5168ceb842f5a3', 'request_id': 'req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:44.737529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a0d03f36dbd4ecf9e5168ceb842f5a3', 'request_id': 'req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:44.821276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a0d03f36dbd4ecf9e5168ceb842f5a3', 'request_id': 'req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:44.837313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] The token cache is being invalidate because project 5a0d03f36dbd4ecf9e5168ceb842f5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:44.837675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:44.838521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:44.840463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 865/1745] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:44 2022] DELETE /identity/v3/projects/5a0d03f36dbd4ecf9e5168ceb842f5a3 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:44.844982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:44.855695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c1edf-a5fa-4bac-9a59-d6c3d84f3e3c service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:44.928133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:44.928685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:44.928962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:44.929231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] PATH_INFO: `/v3/users/b91a5e75391843c9a26473512e373912` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:44.930407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91a5e75391843c9a26473512e373912)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:44.932362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:45.216374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91a5e75391843c9a26473512e373912'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:45.378327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91a5e75391843c9a26473512e373912'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:45.378951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91a5e75391843c9a26473512e373912'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:45.393626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91a5e75391843c9a26473512e373912'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:45.394334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2373ee1-3b87-435a-90c4-0ee74dddb5d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91a5e75391843c9a26473512e373912'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:45.406725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 881/1746] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:44 2022] DELETE /identity/v3/users/b91a5e75391843c9a26473512e373912 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:45.936974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:45.940239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:45.956183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f4be-da96-433a-a7af-f9ecf4cbbfa3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.025004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.025656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.025935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.026227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] PATH_INFO: `/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.027377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e9d99ab5ca4034ace4e71bcbc61469)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.029651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.054762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:46.151864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e9d99ab5ca4034ace4e71bcbc61469', 'request_id': 'req-05e84fc4-aeb3-472d-b952-309404bee8c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:46.159930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e9d99ab5ca4034ace4e71bcbc61469', 'request_id': 'req-05e84fc4-aeb3-472d-b952-309404bee8c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:46.232481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e9d99ab5ca4034ace4e71bcbc61469', 'request_id': 'req-05e84fc4-aeb3-472d-b952-309404bee8c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:46.245779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] The token cache is being invalidate because project 99e9d99ab5ca4034ace4e71bcbc61469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:46.245779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:46.245779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-05e84fc4-aeb3-472d-b952-309404bee8c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:46.247463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 866/1747] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:45 2022] DELETE /identity/v3/projects/99e9d99ab5ca4034ace4e71bcbc61469 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:46.253229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.253621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.253795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.263144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:46.278508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:46.279779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:46.343021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 882/1748] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:46.351411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.351692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.351960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.373495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:46.377130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:46.383108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 867/1749] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:46.387427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.399753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c3d46c-8f39-496c-8e46-f9825bf0dc9a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.404462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.405062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.405338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.405610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.406291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:46.406617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.408255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.420458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 883/1750] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:46.431767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.441604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0d1b49-43fb-46e6-9296-8b4cffce6317 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.446289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.446887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.447178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.447458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.448099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.450014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.610962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 868/1751] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:46 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:46.615678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.625584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f400503-f75b-4fb4-937d-925299c2213c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.685234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.685849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.686125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.686396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.687098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.689312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.867930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 884/1752] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:46 2022] POST /identity/v3/users => generated 294 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:46.872355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.882995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc104540-9ef1-465d-9b8e-aceb8366f412 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.887738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.890417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.890417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.890417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.890417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.892023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.902690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 869/1753] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:46.906776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.917507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2a1bc-976c-42a7-9f0b-dba74fefabe5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.922087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.922680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.924912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.924912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.924912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.925435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b67014f-86f5-4dab-b45a-c1b9804c980d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:46.936349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 885/1754] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:46.940957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:46.951364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe0c8f-59e4-4961-8d66-ed5c999a1b29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:46.956018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:46.956666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:46.956941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:46.957239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] PATH_INFO: `/v3/projects/ee1d89fbb1d3412e958fc40047476b0f/users/7155d4a1b7224f6fb29d2a9d46b9bbbc/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:46.972542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee1d89fbb1d3412e958fc40047476b0f, user_id=7155d4a1b7224f6fb29d2a9d46b9bbbc, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:46.976431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:47.071317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 870/1755] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:46 2022] PUT /identity/v3/projects/ee1d89fbb1d3412e958fc40047476b0f/users/7155d4a1b7224f6fb29d2a9d46b9bbbc/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:47.081889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:47.082170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:47.082441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:47.112337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] MFA Rules not processed for user `7155d4a1b7224f6fb29d2a9d46b9bbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:47.113567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:47.179986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 886/1756] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:47.184473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:47.230878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:47.257310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0f4a-36ea-4c51-a60e-01a6a346cf20 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:47.267796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:47.268513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:47.268854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:47.269199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:47.269915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:47.272873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:47.408092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 871/1757] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:47 2022] POST /identity/v3/projects => generated 385 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:47.414261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:47.426646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f300f8-610c-4e13-8efc-200c70839507 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:47.494748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:47.495321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:47.495602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:47.495876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:47.496524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:47.498472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:48.801287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 887/1758] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:47 2022] POST /identity/v3/users => generated 299 bytes in 1388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:48.807280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:48.824282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda59f64-46c9-49dd-bcbc-3062fb3defbf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:48.829476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:48.830109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:48.830393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:48.830690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:48.831381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:48.834144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:48.847633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 872/1759] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:48.852142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:48.868627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358eb39-c48d-4d89-83a7-646fcadc2150 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:48.876006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:48.877276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:48.877746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:48.878215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] PATH_INFO: `/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a/users/aabb30a76a50453db6d327644572889a/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:48.905661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8dba6877f8451095cc9e94d65b0e3a, user_id=aabb30a76a50453db6d327644572889a, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:48.910977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:49.035795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 888/1760] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:48 2022] PUT /identity/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a/users/aabb30a76a50453db6d327644572889a/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:49.035972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:49.046329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec6f9aa-e971-4c43-9cae-9b05270723da admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:49.061020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:49.119772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:49.120712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:49.120712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:49.120885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:49.121525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:49.123189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6757-bfa1-4cc8-9a11-7b3127576833 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:49.135876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 873/1761] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:49.140041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:49.155985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc034c29-ec05-4e53-87af-ff213a8d7c40 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:49.162904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:49.163707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:49.163991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:49.164273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] PATH_INFO: `/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a/users/aabb30a76a50453db6d327644572889a/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:49.166304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8dba6877f8451095cc9e94d65b0e3a, user_id=aabb30a76a50453db6d327644572889a, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:49.169110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:49.192946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 889/1762] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:49 2022] PUT /identity/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a/users/aabb30a76a50453db6d327644572889a/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:49.205343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:49.205695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:49.205982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:49.246172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] MFA Rules not processed for user `aabb30a76a50453db6d327644572889a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:49.247469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:49.334497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 874/1763] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:49.346230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:49.349623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:49.444936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb853478-f31f-4881-8da3-a07309b8aade admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:49.444936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:49.444936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:49.444936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:49.445305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:49.458429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:49.460204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1200a-484e-4223-ab0a-266bf21d4b1e service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:49.470722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 890/1764] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:49.632950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:49.723982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f89ff-fdd6-4bfa-bca7-10e55cd26857 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:49.731131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:49.732419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:49.732888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:49.733378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] PATH_INFO: `/v3/users/7155d4a1b7224f6fb29d2a9d46b9bbbc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:49.734705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7155d4a1b7224f6fb29d2a9d46b9bbbc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:49.736958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:49.935026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:49.943918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7155d4a1b7224f6fb29d2a9d46b9bbbc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:49.944687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7155d4a1b7224f6fb29d2a9d46b9bbbc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:50.097184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7155d4a1b7224f6fb29d2a9d46b9bbbc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:50.106887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7155d4a1b7224f6fb29d2a9d46b9bbbc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:50.121432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7155d4a1b7224f6fb29d2a9d46b9bbbc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:50.128080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 875/1765] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:49 2022] DELETE /identity/v3/users/7155d4a1b7224f6fb29d2a9d46b9bbbc => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:50.133016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.133498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.133588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.155813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:50.157056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.216104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:50.273383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 891/1766] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:50.282599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.285335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.373471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:50.373663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.421349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8f344-06b2-4e4d-a364-4dbd29e74068 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.426248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.427305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.427598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.427875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.431642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:50.475236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a52f2-3b55-41ce-9240-1339506edde5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:50.481251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:50.486601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:50.486601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:50.486601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:50.496989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.499296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.525979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 892/1767] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:50.542495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:50.545258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:50.574309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 876/1768] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:51.549317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.551501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:51.565625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c3633-c903-4bb7-afbd-51bece25db78 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.572718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.573712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.574267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.574610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] PATH_INFO: `/v3/projects/ee1d89fbb1d3412e958fc40047476b0f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:51.575891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee1d89fbb1d3412e958fc40047476b0f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:51.587104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:51.780286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee1d89fbb1d3412e958fc40047476b0f', 'request_id': 'req-41009b60-9f26-4009-b0aa-7f75261d86b8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:51.788048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee1d89fbb1d3412e958fc40047476b0f', 'request_id': 'req-41009b60-9f26-4009-b0aa-7f75261d86b8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:51.881540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee1d89fbb1d3412e958fc40047476b0f', 'request_id': 'req-41009b60-9f26-4009-b0aa-7f75261d86b8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:51.893868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] The token cache is being invalidate because project ee1d89fbb1d3412e958fc40047476b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:51.894071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:51.894857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:51.898211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 893/1769] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:51 2022] DELETE /identity/v3/projects/ee1d89fbb1d3412e958fc40047476b0f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:51.902186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:51.914647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d6f-94db-4041-bb7b-01c76b3f3f8c service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:51.994888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:51.996308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:51.997438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:51.998046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] PATH_INFO: `/v3/users/aabb30a76a50453db6d327644572889a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:52.000570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabb30a76a50453db6d327644572889a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:52.003834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:52.023331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:52.328639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabb30a76a50453db6d327644572889a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:52.330119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabb30a76a50453db6d327644572889a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:52.453179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabb30a76a50453db6d327644572889a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:52.461826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabb30a76a50453db6d327644572889a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:52.474672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fbe87939-ed51-4cac-9e2f-d00032387ed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabb30a76a50453db6d327644572889a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:52.480970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 877/1770] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:51 2022] DELETE /identity/v3/users/aabb30a76a50453db6d327644572889a => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:53.009680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.016008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.025927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41009b60-9f26-4009-b0aa-7f75261d86b8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.108267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.109058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.109263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.109559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] PATH_INFO: `/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.110810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8dba6877f8451095cc9e94d65b0e3a)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.112770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.140766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:53.289320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8dba6877f8451095cc9e94d65b0e3a', 'request_id': 'req-cac2c511-f9e7-48ba-b7be-81f403ccdccd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:53.296410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8dba6877f8451095cc9e94d65b0e3a', 'request_id': 'req-cac2c511-f9e7-48ba-b7be-81f403ccdccd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:53.433886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8dba6877f8451095cc9e94d65b0e3a', 'request_id': 'req-cac2c511-f9e7-48ba-b7be-81f403ccdccd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:53.457919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] The token cache is being invalidate because project ea8dba6877f8451095cc9e94d65b0e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:53.459769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:53.461180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cac2c511-f9e7-48ba-b7be-81f403ccdccd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:53.466658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 894/1771] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:53 2022] DELETE /identity/v3/projects/ea8dba6877f8451095cc9e94d65b0e3a => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:53.474333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.475633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.477613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.531398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.533164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.647907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 878/1772] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:53.658049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.658942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.659351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.687918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:53.689781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:53.701059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 895/1773] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:53.706001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.738334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdbc9b1-642b-4f8c-b098-49bb5254834f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.750939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.751368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.755685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.755685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.755685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:53.755685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.755685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.784639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 879/1774] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:33:53.789778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.803818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4728408-cc25-4c6f-98c8-b28dd62c3cb7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:53.818815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:53.822257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:53.823251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:53.823944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:53.825418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:53.828346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:53.978617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 896/1775] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:53.983817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:53.996902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824e77b-9cae-417e-bd7a-13fbc61e4c4c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.109590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.110386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.110895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.131357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.131357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.131357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165f41a3-a84a-4672-8aaf-d51cbe14be8f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.233268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 880/1776] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:53 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:54.237022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.267027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c095-a248-4822-bb26-851a44d60f19 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.269997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c095-a248-4822-bb26-851a44d60f19 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.274714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c095-a248-4822-bb26-851a44d60f19 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.274843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef3d9-96f6-4e3a-96be-35f24bb6ce80 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.276440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 881/1777] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:33:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:33:54.281595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.282547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.283028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.283516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.284374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.287298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.292048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.292048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.292048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.309474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 897/1778] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:54.310766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.319966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.322347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:54.326421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.334073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:54.374101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffffdc8-e49c-43f8-9fd7-dd2354cb6bbb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.410686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.416961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.416961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.416961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] PATH_INFO: `/v3/users/c2c50b74a2d34acd9656b40596682d13` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.421668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c50b74a2d34acd9656b40596682d13)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.428638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.458855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 882/1779] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:54.459583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.472938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0366de8a-1536-4fef-91f4-6b9aa1bbcb45 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.480761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.481398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.481653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.481926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.486068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.486068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.504345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 883/1780] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:54.505663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.519137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54783e82-127e-4fbd-9f93-568d71f8d492 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:54.526588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:54.527515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.528406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.528841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] PATH_INFO: `/v3/projects/b63b9e25efb34fadab22a7d4466fac16/users/1c2443818bd9452fa3010daf9193dab0/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.572549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63b9e25efb34fadab22a7d4466fac16, user_id=1c2443818bd9452fa3010daf9193dab0, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:54.576015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285f299-a66a-4e05-849f-fa96c0f3d7f8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:54.721194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 884/1781] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:54 2022] PUT /identity/v3/projects/b63b9e25efb34fadab22a7d4466fac16/users/1c2443818bd9452fa3010daf9193dab0/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:54.727877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c50b74a2d34acd9656b40596682d13'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:54.733846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.734110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.734387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.777845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] MFA Rules not processed for user `1c2443818bd9452fa3010daf9193dab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:54.779662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.832661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c50b74a2d34acd9656b40596682d13'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:54.834710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c50b74a2d34acd9656b40596682d13'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:54.864176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c50b74a2d34acd9656b40596682d13'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:54.867502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-730fee30-0f31-4a1d-aee7-44fd6d1002db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c50b74a2d34acd9656b40596682d13'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:54.883329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 885/1782] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:54.898692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:54.898692 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.899211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 898/1783] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:54 2022] DELETE /identity/v3/users/c2c50b74a2d34acd9656b40596682d13 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:54.907354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:54.907646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:54.908006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:54.930543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:54.931962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:54.999731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:55.011573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236b774-1a67-4877-8be9-efe353421a0a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:55.027344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 899/1784] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:55.028210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:55.029605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:55.102937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3817d46-bce7-4d84-8e94-801e1149f3d7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:55.103677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:55.104357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:55.104652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:55.104920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:55.107332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:55.107868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:55.108144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:55.108635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:55.111250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:55.113850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:55.139856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:55.139856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:55.141524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 886/1785] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:33:55.142565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:55.144386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:55.193442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:55.196999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:55.208990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 900/1786] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:55.217366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71022b-0d25-4cfd-8372-c252cc368913 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:55.221248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:55.221960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:55.222277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:55.222618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:55.235437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:55.237112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:55.252209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 887/1787] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:55.733666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:55.735929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:55.754549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244fd1b-4fed-437c-b0e6-a87120f02e2d service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:55.759957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:55.760598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:55.760868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:55.761135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] PATH_INFO: `/v3/projects/762f35bfd1fa430ab8dcf86358ce0225` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:55.762361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762f35bfd1fa430ab8dcf86358ce0225)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:55.764371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:55.971075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762f35bfd1fa430ab8dcf86358ce0225', 'request_id': 'req-3320ab64-4ef3-4176-992b-31c22a72adc4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:55.978707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762f35bfd1fa430ab8dcf86358ce0225', 'request_id': 'req-3320ab64-4ef3-4176-992b-31c22a72adc4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.095804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762f35bfd1fa430ab8dcf86358ce0225', 'request_id': 'req-3320ab64-4ef3-4176-992b-31c22a72adc4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.128361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] The token cache is being invalidate because project 762f35bfd1fa430ab8dcf86358ce0225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:56.131251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.131251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.135290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 901/1788] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:55 2022] DELETE /identity/v3/projects/762f35bfd1fa430ab8dcf86358ce0225 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:56.136353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:56.150908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2fa5a-7aca-4f3c-a08e-340332d9705c service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:56.220748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:56.221814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:56.222345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:56.222807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] PATH_INFO: `/v3/users/5ca1cfcd6e074ba28ba858a4e626d592` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:56.225621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca1cfcd6e074ba28ba858a4e626d592)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:56.229466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:56.236677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:56.547764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca1cfcd6e074ba28ba858a4e626d592'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.547764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca1cfcd6e074ba28ba858a4e626d592'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.669077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:56.669187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca1cfcd6e074ba28ba858a4e626d592'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.669294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:56.678475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca1cfcd6e074ba28ba858a4e626d592'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.684931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3320ab64-4ef3-4176-992b-31c22a72adc4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:56.697900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:56.698649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:56.698989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:56.703370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] PATH_INFO: `/v3/users/cb847ab3431546d4bb89842150341af8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:56.703370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb847ab3431546d4bb89842150341af8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:56.705358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:56.715797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2191bb25-b55d-42d3-8d91-0a7d99506096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca1cfcd6e074ba28ba858a4e626d592'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:56.731014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 888/1789] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:56 2022] DELETE /identity/v3/users/5ca1cfcd6e074ba28ba858a4e626d592 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:56.955946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb847ab3431546d4bb89842150341af8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.064948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb847ab3431546d4bb89842150341af8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.066057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb847ab3431546d4bb89842150341af8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.080629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb847ab3431546d4bb89842150341af8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.081468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb847ab3431546d4bb89842150341af8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.100363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 902/1790] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:56 2022] DELETE /identity/v3/users/cb847ab3431546d4bb89842150341af8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:33:57.104498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:57.104792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:57.105069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:57.126053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:57.127320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:57.203719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 889/1791] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:57.214903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:57.216531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:57.300178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f98d38-eda7-478d-a73e-d43ca696912e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:57.322739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:57.325131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:57.325838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:57.326216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:57.326563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:57.340522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:57.342202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c088d-0d84-4376-aec9-ac58cd955c3d service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:57.361793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:57.363432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:57.369725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 903/1792] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:33:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:33:57.382702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39a5ed-0328-49df-b8ee-7fd9a36912f8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:57.407660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:57.408907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:57.409402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:57.410014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] PATH_INFO: `/v3/projects/dcafe1965b134d64bd15d3aec03df12d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:57.413639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcafe1965b134d64bd15d3aec03df12d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:57.416114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:57.584516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:57.604855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcafe1965b134d64bd15d3aec03df12d', 'request_id': 'req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.611019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcafe1965b134d64bd15d3aec03df12d', 'request_id': 'req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.766355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcafe1965b134d64bd15d3aec03df12d', 'request_id': 'req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.789115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] The token cache is being invalidate because project dcafe1965b134d64bd15d3aec03df12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:57.789443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.790115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ad2fd4f-618e-43b7-81b8-cddb4614ffc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:57.791749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 890/1793] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:57 2022] DELETE /identity/v3/projects/dcafe1965b134d64bd15d3aec03df12d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:57.798776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:57.799064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:57.799365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:57.829404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:57.830887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:57.931498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 904/1794] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:33:57.933965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:57.934832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:57.935112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:57.958755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:57.960541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:57.969502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 891/1795] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:57.975106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:57.990137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b779066-2bf1-4bca-b089-b6f4bd5d522a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:57.999242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.000205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.000727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.002932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.002932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:33:58.002932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.004366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.022308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 905/1796] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:33:58.026957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.050189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584723-47ae-4805-9356-bf0bf544e8fa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.056358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.056918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.057192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.058310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.058425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.060883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.082030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.085884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:58.123723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bbca9-5d91-435c-a575-971a3ef137c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.123723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.123723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.124068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.124411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] PATH_INFO: `/v3/projects/43245dd76e294fc1b8f2a3b393c54f11` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.126228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43245dd76e294fc1b8f2a3b393c54f11)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.127764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.190743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 892/1797] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:58 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:58.191945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.261797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593ccd3-c445-4552-bd6c-aff1734ab89c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.266808 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.267465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.267741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.268010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.268776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.271452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.302598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43245dd76e294fc1b8f2a3b393c54f11', 'request_id': 'req-7a10143f-f717-420b-a5d3-0cf6ffea4367'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.309527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43245dd76e294fc1b8f2a3b393c54f11', 'request_id': 'req-7a10143f-f717-420b-a5d3-0cf6ffea4367'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.468059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43245dd76e294fc1b8f2a3b393c54f11', 'request_id': 'req-7a10143f-f717-420b-a5d3-0cf6ffea4367'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.475006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 893/1798] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:58 2022] POST /identity/v3/users => generated 305 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:58.479253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.485276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] The token cache is being invalidate because project 43245dd76e294fc1b8f2a3b393c54f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:58.485640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.486582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.488891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 906/1799] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:58 2022] DELETE /identity/v3/projects/43245dd76e294fc1b8f2a3b393c54f11 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:58.490651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b593bdfc-704d-4c8f-b42d-ee6b2f38b8e2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.493283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.494891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:58.557012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.557702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.557991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.558266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.558934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.560633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.574463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 894/1800] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:58.578155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.584592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:58.593375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10143f-f717-420b-a5d3-0cf6ffea4367 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.601685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.601979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.602491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.603074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] PATH_INFO: `/v3/users/4a9a4a38864540bd9ea690123b5d58c9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.605285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9a4a38864540bd9ea690123b5d58c9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.608302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.614364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa282ead-8dcb-4e1b-aad7-0df2cb14db8c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.632322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.634211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.634691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.635784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.636181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.640650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.652341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 895/1801] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:58.657003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.669941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0f404-56d2-4063-ac6d-f0305519e2db admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:58.675667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:58.676313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.677477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.677601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] PATH_INFO: `/v3/projects/174727b395da4628bb0e0e892216b70c/users/3af9b4e9239541c59727aab3c7e3b198/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.704848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=174727b395da4628bb0e0e892216b70c, user_id=3af9b4e9239541c59727aab3c7e3b198, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:58.709974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5915a4-6de3-4b9d-8ab9-3631d4435397 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:58.826502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 896/1802] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:58 2022] PUT /identity/v3/projects/174727b395da4628bb0e0e892216b70c/users/3af9b4e9239541c59727aab3c7e3b198/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:58.839681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:58.839681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:58.839681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:58.871110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] MFA Rules not processed for user `3af9b4e9239541c59727aab3c7e3b198`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:58.872498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:58.894962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9a4a38864540bd9ea690123b5d58c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.945451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 897/1803] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:58 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:33:58.950064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:58.976782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9a4a38864540bd9ea690123b5d58c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.977436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9a4a38864540bd9ea690123b5d58c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.991611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9a4a38864540bd9ea690123b5d58c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:58.992677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9a4a38864540bd9ea690123b5d58c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.006941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 907/1804] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:58 2022] DELETE /identity/v3/users/4a9a4a38864540bd9ea690123b5d58c9 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:59.021926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1acdce-cfe9-426d-8f01-ca11f1c9390c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.084639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.085436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.085869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.086239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.087000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.089639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.202990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 898/1805] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:33:58 2022] POST /identity/v3/projects => generated 409 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:33:59.207491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.209257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:59.222270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516b78a-40df-4400-add8-b7fb8997493c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.300150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.300150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.300150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.300150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.300903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.301190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.439382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 908/1806] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:33:59 2022] POST /identity/v3/users => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:33:59.444667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.455170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefed8df-0d36-490e-9211-bb50a4484c6c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.460007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.460610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.460890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.461159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.461827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.463485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.476801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 899/1807] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:59.482743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.494071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843caaf-9ac9-42ae-b4d6-6958b38590d6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.499090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.499794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.500063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.500375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] PATH_INFO: `/v3/projects/6d2d19281d6c4fd38e96872118f1a572/users/e46be47059f44fdc83f2159c0007f4ac/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.520757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2d19281d6c4fd38e96872118f1a572, user_id=e46be47059f44fdc83f2159c0007f4ac, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.523657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.625859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.628644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:59.650267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74025fae-9ad0-4469-8f32-dc2c326169d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.658205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.659185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.659664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.660035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] PATH_INFO: `/v3/projects/5a874f6a61134c138d582b0a84358df0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.662877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a874f6a61134c138d582b0a84358df0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.665696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.694210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 909/1808] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:59 2022] PUT /identity/v3/projects/6d2d19281d6c4fd38e96872118f1a572/users/e46be47059f44fdc83f2159c0007f4ac/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:59.694210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.764453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:33:59.765747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae37a6-b8b5-4cff-93b3-c48ab5ddba96 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.771972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.772599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.780273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.780273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.780273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.780273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.790945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 910/1809] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:33:59.794906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.815097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba34fd83-4794-4740-b5af-4f8832051f00 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:33:59.838024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:33:59.838973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.839439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.839977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] PATH_INFO: `/v3/projects/6d2d19281d6c4fd38e96872118f1a572/users/e46be47059f44fdc83f2159c0007f4ac/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.843314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d2d19281d6c4fd38e96872118f1a572, user_id=e46be47059f44fdc83f2159c0007f4ac, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:33:59.848256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a874f6a61134c138d582b0a84358df0', 'request_id': 'req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.849255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d25dc-33a1-4746-90d2-29eac7260eb1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:33:59.860401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a874f6a61134c138d582b0a84358df0', 'request_id': 'req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.877447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 911/1810] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:33:59 2022] PUT /identity/v3/projects/6d2d19281d6c4fd38e96872118f1a572/users/e46be47059f44fdc83f2159c0007f4ac/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:59.889737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:33:59.890578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:33:59.890578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:33:59.940458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] MFA Rules not processed for user `e46be47059f44fdc83f2159c0007f4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:33:59.943666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:33:59.952518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a874f6a61134c138d582b0a84358df0', 'request_id': 'req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.969000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] The token cache is being invalidate because project 5a874f6a61134c138d582b0a84358df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:33:59.969752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.975682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:33:59.985945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 900/1811] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:33:59 2022] DELETE /identity/v3/projects/5a874f6a61134c138d582b0a84358df0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:33:59.991088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:33:59.992731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:00.052475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 912/1812] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:33:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:00.063510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:00.065223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:00.073862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fe8c9-63e1-44c6-8e6d-c0ae97cf4bea admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:00.128026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb8631-422a-4ab6-b64a-d0281c667d34 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:00.133974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:00.134772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:00.135123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:00.135271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:00.135861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:00.136033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:00.136130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:00.136388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] PATH_INFO: `/v3/users/d34006d2f81342fc8240c98b5b1d565c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:00.137596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d34006d2f81342fc8240c98b5b1d565c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:00.139594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:00.147993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:00.149535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:00.163924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 913/1813] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:00.412998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd34006d2f81342fc8240c98b5b1d565c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:00.413762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd34006d2f81342fc8240c98b5b1d565c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:00.416375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:00.530451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd34006d2f81342fc8240c98b5b1d565c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:00.540941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd34006d2f81342fc8240c98b5b1d565c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:00.566757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd34006d2f81342fc8240c98b5b1d565c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:00.572901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 901/1814] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:33:59 2022] DELETE /identity/v3/users/d34006d2f81342fc8240c98b5b1d565c => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:00.832035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:00.843851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd503b9-5ab6-407b-8cac-6f864bf32e58 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:00.912301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:00.912840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:00.913282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:00.913700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:00.915734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:00.991436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:00.993181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76edd3-afd3-47f2-ab1e-dcb73c6399db service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:01.003954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 914/1815] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:01.087659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:01.088590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:01.167047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294ddbe-aa0e-4244-994e-595b0c803c26 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:01.170595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:01.171275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.171557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.171831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] PATH_INFO: `/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.172970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8e7b3078a741f7bb8dc963783a4e00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:01.175019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:01.347469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8e7b3078a741f7bb8dc963783a4e00', 'request_id': 'req-b7df0087-c0ac-4202-96a9-492e41f194ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:01.353812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8e7b3078a741f7bb8dc963783a4e00', 'request_id': 'req-b7df0087-c0ac-4202-96a9-492e41f194ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:01.498688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8e7b3078a741f7bb8dc963783a4e00', 'request_id': 'req-b7df0087-c0ac-4202-96a9-492e41f194ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:01.510830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] The token cache is being invalidate because project dc8e7b3078a741f7bb8dc963783a4e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:01.511357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:01.512190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b7df0087-c0ac-4202-96a9-492e41f194ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:01.514628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 902/1816] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:01 2022] DELETE /identity/v3/projects/dc8e7b3078a741f7bb8dc963783a4e00 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:01.521966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.522484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.522484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.546057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:01.547365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:01.618910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 915/1817] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:01.627678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.628184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.628274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.651391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:01.652686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:01.662628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 903/1818] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:01.667107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:01.679430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206caaac-047e-45c0-bb22-dcb7264da242 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:01.684318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:01.684958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.685205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.685476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.686250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:01.686553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:01.688184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:01.701842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 916/1819] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:01.706197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:01.723019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16d85e-27d1-41c1-ae7a-9ed9d95c6ecb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:01.729663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:01.730306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.730651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.730990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.731736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:01.733706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:01.894740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 904/1820] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:01 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:01.899039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:01.909220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f57b4-efd6-45f5-9fae-a8065b7250c9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:01.968561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:01.969403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:01.969751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:01.970136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:01.971324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:01.976002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.079237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 917/1821] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:01 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:02.080513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.091319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ae55d-602a-4596-8b3f-31a8978083e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.096371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.097065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.097429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.097800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.098781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.101029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.112119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 905/1822] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:02.116769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.127275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46fa8e-7592-4f29-a57d-f73a7ecf3f1a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.133233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.134000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.134181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.134455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.143059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.143059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50324c-c04c-44b5-8f34-eec6525087dc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.149679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 918/1823] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:02.153929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.165158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d498c6e7-1c24-4264-a542-efcdcc148b01 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.169935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.170607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.170887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.171213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] PATH_INFO: `/v3/projects/fdc42a0024b54e4cbc92a3f0d00b13ac/users/9808f2d12ab04394961289a05cf7e521/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.187350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc42a0024b54e4cbc92a3f0d00b13ac, user_id=9808f2d12ab04394961289a05cf7e521, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.190599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.271777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 906/1824] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:02 2022] PUT /identity/v3/projects/fdc42a0024b54e4cbc92a3f0d00b13ac/users/9808f2d12ab04394961289a05cf7e521/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:02.284273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.284595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.284595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.316907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] MFA Rules not processed for user `9808f2d12ab04394961289a05cf7e521`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:02.318995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:02.399249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:02.421622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 919/1825] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:02.426207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.493912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bfa45-9101-4955-b32f-39931c0781c7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.498630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.499273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.499560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.499826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.500501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.502397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.527479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:02.589091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 907/1826] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:02.592756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.604710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3e2e4-1041-4bcb-99dd-5e27c3862bce None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.664597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.665212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.665488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.665756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.667577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.668372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.771609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 920/1827] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:02.778834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.790900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b9a0-a71d-47a2-a4ed-98c0cbeebd10 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.795479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.796214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.796493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.796761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.797602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.799287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.811474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 908/1828] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:02.819333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.829186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f451ddf-0592-4b86-8d8e-cad77d8466d6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.834190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.834805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.835080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.835387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.836035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.837652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016485d8-a48f-4d18-bd16-140024c6b009 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.848971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 921/1829] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:02.858271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:02.869011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fe7459-8f7a-4205-945d-0a8a72b391bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:02.874911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:02.875711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.876267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.876616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] PATH_INFO: `/v3/projects/423b973dfd4842bfa366b772c26c4ec6/users/14e050feb8a640e38febfa5a78b72c08/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:02.897128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=423b973dfd4842bfa366b772c26c4ec6, user_id=14e050feb8a640e38febfa5a78b72c08, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:02.898892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:02.906878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:02.987663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 909/1830] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:02 2022] PUT /identity/v3/projects/423b973dfd4842bfa366b772c26c4ec6/users/14e050feb8a640e38febfa5a78b72c08/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:02.998867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6ab5-7009-4136-b309-356737611aed None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:02.999174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6ab5-7009-4136-b309-356737611aed None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:02.999457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6ab5-7009-4136-b309-356737611aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.032916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a80e6ab5-7009-4136-b309-356737611aed None None] MFA Rules not processed for user `14e050feb8a640e38febfa5a78b72c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:03.034324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a80e6ab5-7009-4136-b309-356737611aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:03.104361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 922/1831] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:03.109964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.182402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857aac3-9bbd-4320-8f44-2a8a8c66ed4d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.189689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.190608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.191027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.191390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.192202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.194368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.332577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:03.346691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 910/1832] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:03 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:03.348204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e6ab5-7009-4136-b309-356737611aed None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.363245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e6ab5-7009-4136-b309-356737611aed None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.435816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.436650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.437012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.437410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.438588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.444428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.556685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 923/1833] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:03 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:03.561321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.576412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f1a9-0c3c-4d1a-9d6f-3741dae2b217 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.584219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.588131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.588131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.588131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.588131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.588131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.603654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 911/1834] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:03.608336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.619443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:03.621777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3516-e5f3-4f03-a9a4-911f1aba9658 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.629118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.630189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.630904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.631347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] PATH_INFO: `/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c/users/4a65744ff9164660b788e2c592a1b650/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.657547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395ffd4b025344b1bca25ee6ddfbb77c, user_id=4a65744ff9164660b788e2c592a1b650, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.662328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.816395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 924/1835] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:03 2022] PUT /identity/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c/users/4a65744ff9164660b788e2c592a1b650/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:03.822415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.838070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19c621-91ea-4c74-b003-6b7f4b6da03c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.838443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:03.915749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.916395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.916725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.917087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.917755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.919448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acd8b98-fd44-4d1a-9151-0eca142f14a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.931792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 912/1836] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:03.935896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:03.948827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea7c7e-d98b-41a7-982a-9b24af6a5e03 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:03.966114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:03.966114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:03.966114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:03.966114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] PATH_INFO: `/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c/users/4a65744ff9164660b788e2c592a1b650/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:03.966467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395ffd4b025344b1bca25ee6ddfbb77c, user_id=4a65744ff9164660b788e2c592a1b650, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:03.969446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:03.982881 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 925/1837] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:03 2022] PUT /identity/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c/users/4a65744ff9164660b788e2c592a1b650/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:03.999798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.000176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.000524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.055585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] MFA Rules not processed for user `4a65744ff9164660b788e2c592a1b650`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:04.063378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.158548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 913/1838] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:04.171101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:04.173126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:04.206089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:04.241982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ec493-7622-445f-be5d-fd0df73b8c33 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:04.246443 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:04.247063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.247370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.247649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.257774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:04.259247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:04.272366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 926/1839] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:04.530955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:04.541423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116b23-0e51-47aa-83df-be3988366c12 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:04.546357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:04.546977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:04.547285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:04.547553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:04.611720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:04.613468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcfdd17-e2b6-48cd-9c31-82bbf075a7ad service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:04.623524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:04.628086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 914/1840] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:04.991732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:06.220081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:15.879393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=21 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:16.507667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=65 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:19.165851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.265628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd5a0e-d824-463a-925e-a87f37b6f1a6 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.267395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.268240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.268616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.268917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] PATH_INFO: `/v3/users/9808f2d12ab04394961289a05cf7e521` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.270297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9808f2d12ab04394961289a05cf7e521)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.273296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.587349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9808f2d12ab04394961289a05cf7e521'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:19.696022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9808f2d12ab04394961289a05cf7e521'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:19.696798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9808f2d12ab04394961289a05cf7e521'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:19.724415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9808f2d12ab04394961289a05cf7e521'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:19.725670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9808f2d12ab04394961289a05cf7e521'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:19.749826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 927/1841] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:19 2022] DELETE /identity/v3/users/9808f2d12ab04394961289a05cf7e521 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:19.754224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.754729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.754829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.777620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:19.779137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:19.855537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 915/1842] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:19.866090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:19.867721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:19.939093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cedd2b-2b67-491f-9965-c8fbd2abf8f7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:19.943707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:19.944580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:19.944861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:19.945134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:19.960753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:19.964055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:19.983219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 928/1843] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:20.547384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.548875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:20.561878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9aa4-cf2d-4ca1-b89a-496523c62937 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.567564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.568848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.569138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.569474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] PATH_INFO: `/v3/users/3af9b4e9239541c59727aab3c7e3b198` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.570619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3af9b4e9239541c59727aab3c7e3b198)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.572675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.638241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:20.640517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:20.655547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a257f-2968-4271-a5ee-ffaa7572753f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:20.660695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:20.661564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:20.662325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:20.662603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] PATH_INFO: `/v3/projects/fdc42a0024b54e4cbc92a3f0d00b13ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:20.663980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc42a0024b54e4cbc92a3f0d00b13ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:20.666337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:20.691591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:20.900618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc42a0024b54e4cbc92a3f0d00b13ac', 'request_id': 'req-c823b6c2-b04c-49a3-80b8-f3db57909f20'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:20.903932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3af9b4e9239541c59727aab3c7e3b198'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:20.906343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3af9b4e9239541c59727aab3c7e3b198'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:20.912082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc42a0024b54e4cbc92a3f0d00b13ac', 'request_id': 'req-c823b6c2-b04c-49a3-80b8-f3db57909f20'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.068403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3af9b4e9239541c59727aab3c7e3b198'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.081388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3af9b4e9239541c59727aab3c7e3b198'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.105100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b94108dc-b367-4254-b594-62aca235921d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3af9b4e9239541c59727aab3c7e3b198'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.116328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 916/1844] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:20 2022] DELETE /identity/v3/users/3af9b4e9239541c59727aab3c7e3b198 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:21.124064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.124525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.124951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.150862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:21.152247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:21.163475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc42a0024b54e4cbc92a3f0d00b13ac', 'request_id': 'req-c823b6c2-b04c-49a3-80b8-f3db57909f20'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.179317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] The token cache is being invalidate because project fdc42a0024b54e4cbc92a3f0d00b13ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:21.179765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.180696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.183621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 929/1845] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:20 2022] DELETE /identity/v3/projects/fdc42a0024b54e4cbc92a3f0d00b13ac => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:21.187939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:21.191218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:21.246928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 917/1846] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:21.257680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:21.259291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:21.276325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:21.292533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823b6c2-b04c-49a3-80b8-f3db57909f20 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:21.298604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:21.299297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.299578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.300130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] PATH_INFO: `/v3/users/14e050feb8a640e38febfa5a78b72c08` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.304998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e050feb8a640e38febfa5a78b72c08)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.304998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.334269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a728cd-17e7-4f93-8809-9028d1bdfca7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:21.338228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:21.338842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:21.339120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:21.339437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:21.349939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:21.351567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:21.362491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 918/1847] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:21.578777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e050feb8a640e38febfa5a78b72c08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.708308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e050feb8a640e38febfa5a78b72c08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.710237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e050feb8a640e38febfa5a78b72c08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.750114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e050feb8a640e38febfa5a78b72c08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.751647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e050feb8a640e38febfa5a78b72c08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:21.779915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 930/1848] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:21 2022] DELETE /identity/v3/users/14e050feb8a640e38febfa5a78b72c08 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:22.071574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.074822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:22.111652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5997c1-450f-4a6b-b359-b11f6d8e61c0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.245840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.246510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.246792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.247069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] PATH_INFO: `/v3/projects/174727b395da4628bb0e0e892216b70c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.248305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=174727b395da4628bb0e0e892216b70c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.250789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:22.426178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174727b395da4628bb0e0e892216b70c', 'request_id': 'req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.433696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174727b395da4628bb0e0e892216b70c', 'request_id': 'req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.449874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.451372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:22.469890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627e615-f19c-4ced-b466-09cf97946234 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.495783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:22.534261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.534905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.535200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.535474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] PATH_INFO: `/v3/projects/423b973dfd4842bfa366b772c26c4ec6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.536682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423b973dfd4842bfa366b772c26c4ec6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.541851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:22.550554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174727b395da4628bb0e0e892216b70c', 'request_id': 'req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.566512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] The token cache is being invalidate because project 174727b395da4628bb0e0e892216b70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:22.566953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.568431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.570810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 919/1849] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:22 2022] DELETE /identity/v3/projects/174727b395da4628bb0e0e892216b70c => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:22.575605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.577327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:22.599125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4ac44-ace9-4e33-a35b-9709fcfc80ac admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.604775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.605730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.606094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.606470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] PATH_INFO: `/v3/users/e46be47059f44fdc83f2159c0007f4ac` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.607793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46be47059f44fdc83f2159c0007f4ac)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.609922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:22.683991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423b973dfd4842bfa366b772c26c4ec6', 'request_id': 'req-c542488c-615f-410b-809c-1b4f27796a73'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.691876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423b973dfd4842bfa366b772c26c4ec6', 'request_id': 'req-c542488c-615f-410b-809c-1b4f27796a73'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.808033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423b973dfd4842bfa366b772c26c4ec6', 'request_id': 'req-c542488c-615f-410b-809c-1b4f27796a73'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.828537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] The token cache is being invalidate because project 423b973dfd4842bfa366b772c26c4ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:22.828835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.829602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.835189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 931/1850] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:22 2022] DELETE /identity/v3/projects/423b973dfd4842bfa366b772c26c4ec6 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:22.836563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:22.838201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:22.858669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:22.866325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46be47059f44fdc83f2159c0007f4ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.867281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46be47059f44fdc83f2159c0007f4ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:22.918262 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542488c-615f-410b-809c-1b4f27796a73 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:22.923565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:22.924216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:22.924525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:22.924804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] PATH_INFO: `/v3/users/4a65744ff9164660b788e2c592a1b650` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:22.925977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a65744ff9164660b788e2c592a1b650)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:22.929134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:23.079301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46be47059f44fdc83f2159c0007f4ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.088065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46be47059f44fdc83f2159c0007f4ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.101650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46be47059f44fdc83f2159c0007f4ac'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.112034 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 920/1851] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:22 2022] DELETE /identity/v3/users/e46be47059f44fdc83f2159c0007f4ac => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:23.225971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a65744ff9164660b788e2c592a1b650'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.316729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a65744ff9164660b788e2c592a1b650'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.317569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a65744ff9164660b788e2c592a1b650'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.334666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a65744ff9164660b788e2c592a1b650'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.335446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a65744ff9164660b788e2c592a1b650'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:23.353031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 932/1852] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:22 2022] DELETE /identity/v3/users/4a65744ff9164660b788e2c592a1b650 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:23.819256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:23.821623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:23.892145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28b7e0-3709-49f2-94ac-2cf1be28ad9d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:23.951446 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:23.952101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:23.952519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:23.952649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] PATH_INFO: `/v3/projects/6d2d19281d6c4fd38e96872118f1a572` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:23.954082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d2d19281d6c4fd38e96872118f1a572)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:23.957174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:23.961647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:23.963762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:23.979966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1da23-f716-4396-a4a6-7dcfdd9b9b95 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:23.985294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:23.986172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:23.986453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:23.986728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] PATH_INFO: `/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:23.987942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395ffd4b025344b1bca25ee6ddfbb77c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:23.992337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.168896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395ffd4b025344b1bca25ee6ddfbb77c', 'request_id': 'req-098b7abe-bbea-4587-8ed1-c45b98fe6854'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.173514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d2d19281d6c4fd38e96872118f1a572', 'request_id': 'req-e15e0b81-99a7-4db7-bd54-ad4967ddea21'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.179267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395ffd4b025344b1bca25ee6ddfbb77c', 'request_id': 'req-098b7abe-bbea-4587-8ed1-c45b98fe6854'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.181302 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d2d19281d6c4fd38e96872118f1a572', 'request_id': 'req-e15e0b81-99a7-4db7-bd54-ad4967ddea21'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.265609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395ffd4b025344b1bca25ee6ddfbb77c', 'request_id': 'req-098b7abe-bbea-4587-8ed1-c45b98fe6854'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.278517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] The token cache is being invalidate because project 395ffd4b025344b1bca25ee6ddfbb77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:24.278812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.279540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-098b7abe-bbea-4587-8ed1-c45b98fe6854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.281350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 933/1853] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:23 2022] DELETE /identity/v3/projects/395ffd4b025344b1bca25ee6ddfbb77c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:24.287705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297909e-405b-4224-a771-fe559d74cb2f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.288188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297909e-405b-4224-a771-fe559d74cb2f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.288400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297909e-405b-4224-a771-fe559d74cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.310758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9297909e-405b-4224-a771-fe559d74cb2f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:24.312255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9297909e-405b-4224-a771-fe559d74cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:24.364655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d2d19281d6c4fd38e96872118f1a572', 'request_id': 'req-e15e0b81-99a7-4db7-bd54-ad4967ddea21'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.377064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] The token cache is being invalidate because project 6d2d19281d6c4fd38e96872118f1a572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:24.377853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.378455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e15e0b81-99a7-4db7-bd54-ad4967ddea21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:24.380517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 921/1854] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:23 2022] DELETE /identity/v3/projects/6d2d19281d6c4fd38e96872118f1a572 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:24.385834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cf6e-a64a-4e43-a91b-9adf74de4374 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.386109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cf6e-a64a-4e43-a91b-9adf74de4374 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.386384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cf6e-a64a-4e43-a91b-9adf74de4374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.394063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 934/1855] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:24.404345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.404681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.415370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.415610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3d87cf6e-a64a-4e43-a91b-9adf74de4374 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:24.415610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d87cf6e-a64a-4e43-a91b-9adf74de4374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:24.439116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 922/1856] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:24.439116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.439116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.439116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.448763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:24.450048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:24.466473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 935/1857] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:24.467785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.473468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:24.473468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:24.485792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b8c3b8-5921-44ea-9b53-cfc851a5e0f3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.486164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 923/1858] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:24.487267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.493568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.494201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.494481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.494756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.496141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:24.496495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.498257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b420253-791c-47c8-8ba0-04c4145d3a76 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.498396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.503786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.504482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.507674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.507674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.507674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:24.507674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.508922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.512794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 936/1859] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:24.517550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.523844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 924/1860] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:24.529368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:24.529519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.536357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99168c4e-01a4-4f48-b160-6d3aa9b881f9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.540033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.540660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.541015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69808a81-4958-4406-996a-3217b45c3271 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.541160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.541323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.541940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.545139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.546179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.546796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.547071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.547377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.548014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.550155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.621902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:24.688911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 925/1861] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:24.692651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.716862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 937/1862] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:24.720999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.761424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45100cdd-2631-43b1-986b-295f1a147850 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.801742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf8ce6f-e71f-45b1-9d9c-f6f7b7bd2dae admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.810004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.810993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.811492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.811975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.813065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.816986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.820812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.821450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.821757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.822067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.822706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.824675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.943651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 938/1863] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:24.948521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.962112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc4652-2af4-4c41-aac7-59bd6805fcca admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.966865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:24.967520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:24.967800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:24.968069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:24.968751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:24.970381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:24.981783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 939/1864] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:24.986394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:24.996531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 926/1865] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:24 2022] POST /identity/v3/users => generated 284 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:24.996926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a464318-fbbf-479f-84da-ba7a638420f3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:24.997714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.002275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.002888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.003183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.003459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.004509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.006223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.009030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2bff7-9b7c-4339-8e53-1ae7eea82602 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.015226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.015848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.016129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.016402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.017064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.018695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.021028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 940/1866] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:25.026098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.036475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 927/1867] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:25.036841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6b449-efb8-4ec2-8193-e1318d7aea96 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.037999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.041820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.042485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.044134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.044134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] PATH_INFO: `/v3/projects/6cac41b3f9c947d4bf5c6fa292bc8435/users/bc1bff02b080447286dda97594b6f50f/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.048390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940edb0-6fbc-461b-8a04-e6db6f735c91 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.053479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.054050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.054373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.054682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.055384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.057009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.063834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cac41b3f9c947d4bf5c6fa292bc8435, user_id=bc1bff02b080447286dda97594b6f50f, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.066979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00eab3-3f3c-48d8-96f5-f9038703aa56 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.076342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 928/1868] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:25.077522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.089070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9199a8-a4e6-4206-aff4-77a6b36c88c9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.093759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.094417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.094695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.094965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] PATH_INFO: `/v3/projects/5c2379c7f0414c06a32686c44c8f4e39/users/8585669077cd4b399bacbe364cc48fde/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.110838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c2379c7f0414c06a32686c44c8f4e39, user_id=8585669077cd4b399bacbe364cc48fde, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.113502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb9cd3-7635-4ecb-8f07-6c1aa92b21bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.191630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 941/1869] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:25 2022] PUT /identity/v3/projects/6cac41b3f9c947d4bf5c6fa292bc8435/users/bc1bff02b080447286dda97594b6f50f/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:25.204117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.204426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.204709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.233293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 929/1870] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:25 2022] PUT /identity/v3/projects/5c2379c7f0414c06a32686c44c8f4e39/users/8585669077cd4b399bacbe364cc48fde/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:25.241444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.241444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.241444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.241787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] MFA Rules not processed for user `bc1bff02b080447286dda97594b6f50f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:25.241787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:25.274594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] MFA Rules not processed for user `8585669077cd4b399bacbe364cc48fde`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:25.275857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:25.327456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 942/1871] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:25.331732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.367957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 930/1872] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:25.373162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.449551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169a854-938a-4925-8b7a-2b146b13b8e2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.456431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069a15-5d20-40a9-a13a-5add28a803d9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.456573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.456573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.456573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.456573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.456901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.458230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.458462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.459922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.460179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.465031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.465752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.467850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.575789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 943/1873] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/projects => generated 383 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:25.580267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.609473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:25.615866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 931/1874] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:25.619646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.650094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40648c-05ba-4e98-81a0-11e73753e9b2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.686007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:25.708007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2391b7-9a87-4267-b024-197bf4e2d105 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.718742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.719558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.720005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.720282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.721064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.723065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.736170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.737476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.737822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.738166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.738938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.741293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.846925 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 944/1875] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/users => generated 298 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:25.851992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.862639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7fd28-3f62-45ab-bdc2-7d62fb92f94f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.868546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.868718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 932/1876] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:25 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:25.869165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.872057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.872057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.872057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.872057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.875750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.887279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 945/1877] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:25.887461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c20a1-ba97-4ba8-a5de-51c0766644d2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.888454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.891794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.892414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.892695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.892967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.893616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.895252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.898763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871bf232-9acb-4211-8d38-c365ca3223bf admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.903470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.904114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.904387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.904654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] PATH_INFO: `/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5/users/c0e13ee009674bd1b42b153e1e413000/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.905661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 933/1878] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:25.909759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:25.919713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a605f-ebc2-4bcd-b872-db286dc28db1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:25.920498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d9ec897e1d4db5b1367bb4996e1ff5, user_id=c0e13ee009674bd1b42b153e1e413000, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.923098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.924728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:25.925435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:25.926105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:25.926105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] PATH_INFO: `/v3/projects/4b09382991a144769cbfa01387a7cd17/users/92c71830aad44368b4e47733a115f8ab/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:25.944911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b09382991a144769cbfa01387a7cd17, user_id=92c71830aad44368b4e47733a115f8ab, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:25.947721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:25.953218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:26.061117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 946/1879] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:25 2022] PUT /identity/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5/users/c0e13ee009674bd1b42b153e1e413000/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:26.065827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.104700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 934/1880] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:25 2022] PUT /identity/v3/projects/4b09382991a144769cbfa01387a7cd17/users/92c71830aad44368b4e47733a115f8ab/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:26.116537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.140782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce04c2-deb8-4933-939c-f7e5182ad733 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.185259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235593e0-1587-4d8b-a904-811b422767f1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.188764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:26.191422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.192048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.192333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.192605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.194263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.196098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.209809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 935/1881] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:26.210004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.210004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.210004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.210004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.210385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.211586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.211705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.223346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd762d-0eda-46a7-9262-b0496acdb846 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.227122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 947/1882] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:26.232736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.233615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.234604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.234937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] PATH_INFO: `/v3/projects/4b09382991a144769cbfa01387a7cd17/users/92c71830aad44368b4e47733a115f8ab/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.236721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.237126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b09382991a144769cbfa01387a7cd17, user_id=92c71830aad44368b4e47733a115f8ab, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.240453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577dcb6-10ab-4032-a415-6975fe669abc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.249004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b950de-9a55-4a55-a979-4d654ce8cb53 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.264466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 936/1883] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:26 2022] PUT /identity/v3/projects/4b09382991a144769cbfa01387a7cd17/users/92c71830aad44368b4e47733a115f8ab/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:26.267083 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.267789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.268489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.302384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] MFA Rules not processed for user `92c71830aad44368b4e47733a115f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:26.304615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:26.315707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.316378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.316655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.316929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] PATH_INFO: `/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5/users/c0e13ee009674bd1b42b153e1e413000/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.318943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d9ec897e1d4db5b1367bb4996e1ff5, user_id=c0e13ee009674bd1b42b153e1e413000, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.321737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531def50-e9dd-4ca6-825f-8f2db9409ea2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.333053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 948/1884] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:26 2022] PUT /identity/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5/users/c0e13ee009674bd1b42b153e1e413000/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:26.344378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.344637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.344913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.382251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] MFA Rules not processed for user `c0e13ee009674bd1b42b153e1e413000`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:26.383475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:26.403315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 937/1885] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:26.420301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.423497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:26.502611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979d491-be99-459e-a8ab-d662757f2839 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.506220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 949/1886] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:26.518998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.520416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:26.525309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:26.578850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.580412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.581148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.582521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.598481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d7c01-3e47-476a-8fe8-2f49100430d1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.602661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.603339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.603616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.603892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.614211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.617053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.628488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 950/1887] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:26.694868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:26.696493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:26.707501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 938/1888] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:26.903491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:26.910122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac444d-86a7-4146-b583-facf5c5a6c3a service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:26.922088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:26.922733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:26.923006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:26.923971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:26.960248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:27.081135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:27.087341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:27.102668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 951/1889] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:27.155185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:27.164513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d83732-03c8-483d-a5cb-ebf405b82b78 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:27.169113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:27.169828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:27.170156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:27.170489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:27.243456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:27.245117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0db06aa-db96-474f-abe8-b27100e5cb15 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:27.255038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 939/1890] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:27.536726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:28.571574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:28.637977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52677ab9-461d-401a-ae20-60e0d506aa25 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:28.642583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:28.643234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:28.643513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:28.643780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] PATH_INFO: `/v3/users/bc1bff02b080447286dda97594b6f50f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:28.644900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1bff02b080447286dda97594b6f50f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:28.647835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:28.946312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:28.952121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1bff02b080447286dda97594b6f50f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.065512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1bff02b080447286dda97594b6f50f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.066625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1bff02b080447286dda97594b6f50f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.095097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1bff02b080447286dda97594b6f50f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.096234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1bff02b080447286dda97594b6f50f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:29.110224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 952/1891] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:28 2022] DELETE /identity/v3/users/bc1bff02b080447286dda97594b6f50f => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:29.117526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.118069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.118277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.140506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:29.142804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.190547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:29.229835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 940/1892] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:29.241002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:29.243388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.346783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0395a-21c5-4881-838e-5a77ee9ea592 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:29.351865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:29.353239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.353239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.353239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.363440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:29.365535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:29.376115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 953/1893] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:29.969824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:29.971441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:29.986293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c196c8-55ea-41a9-878a-6dec523a2d10 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:29.992506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:29.993127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:29.993444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:29.993764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] PATH_INFO: `/v3/projects/6cac41b3f9c947d4bf5c6fa292bc8435` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:29.994948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cac41b3f9c947d4bf5c6fa292bc8435)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:29.997689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:30.120467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cac41b3f9c947d4bf5c6fa292bc8435', 'request_id': 'req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.128023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cac41b3f9c947d4bf5c6fa292bc8435', 'request_id': 'req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.236204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cac41b3f9c947d4bf5c6fa292bc8435', 'request_id': 'req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.270139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] The token cache is being invalidate because project 6cac41b3f9c947d4bf5c6fa292bc8435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:30.270484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.271278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.273071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 941/1894] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:29 2022] DELETE /identity/v3/projects/6cac41b3f9c947d4bf5c6fa292bc8435 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:30.277266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:30.289447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738111f3-cafe-4cdf-8822-d22194d9a1c0 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:30.365502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:30.366580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:30.366868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:30.367173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] PATH_INFO: `/v3/users/c0e13ee009674bd1b42b153e1e413000` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:30.368518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0e13ee009674bd1b42b153e1e413000)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:30.370961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:30.625778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0e13ee009674bd1b42b153e1e413000'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.728453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0e13ee009674bd1b42b153e1e413000'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.729568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0e13ee009674bd1b42b153e1e413000'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.743571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:30.749560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0e13ee009674bd1b42b153e1e413000'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.751512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-579b310d-1444-4840-9628-e7bed23ba565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0e13ee009674bd1b42b153e1e413000'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:30.767792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 954/1895] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:30 2022] DELETE /identity/v3/users/c0e13ee009674bd1b42b153e1e413000 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:31.293535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.295122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.321761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ffdc4-b1e8-41c8-8995-0b07b4d066f3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.382364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.383047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.383400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.383721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] PATH_INFO: `/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.384936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d9ec897e1d4db5b1367bb4996e1ff5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.386926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.621254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d9ec897e1d4db5b1367bb4996e1ff5', 'request_id': 'req-df089578-6b5f-4d5e-85e4-32771f7f6393'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:31.642313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d9ec897e1d4db5b1367bb4996e1ff5', 'request_id': 'req-df089578-6b5f-4d5e-85e4-32771f7f6393'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:31.765070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d9ec897e1d4db5b1367bb4996e1ff5', 'request_id': 'req-df089578-6b5f-4d5e-85e4-32771f7f6393'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:31.771805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:31.781247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] The token cache is being invalidate because project 24d9ec897e1d4db5b1367bb4996e1ff5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:31.781746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:31.782516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df089578-6b5f-4d5e-85e4-32771f7f6393 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:31.788418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 942/1896] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:31 2022] DELETE /identity/v3/projects/24d9ec897e1d4db5b1367bb4996e1ff5 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:31.790048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.790508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.790600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.831074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:31.831074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.889757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 955/1897] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:31.898707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.899134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.899572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.920748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:31.921979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:31.931109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 943/1898] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:31.935364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.946789 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b4195-7a90-48b1-b953-7579e4973183 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.951730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.952360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.952642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.952924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.953608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:31.954735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.955712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:31.971205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 956/1899] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:31.976430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:31.987049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d0809-9f04-4abe-acfa-1a14eda2740f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:31.992608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:31.993224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:31.993668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:31.993999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:31.994629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:31.996613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.144837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 944/1900] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:31 2022] POST /identity/v3/projects => generated 359 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:32.149560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.160306 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e87ea-60f0-41a7-98f8-7b8c135fb578 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.253330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.253994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.254302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.254600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.255350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.259316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.390136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 957/1901] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:32 2022] POST /identity/v3/users => generated 280 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:32.394341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.406182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af0995-8bfa-4009-9935-ab3aa2a6712d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.420619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.422588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.423385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.429545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.429545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.429545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.457287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 945/1902] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:32.457515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.481489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98be1500-8cc6-4361-aa47-dfcfb6c37d94 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.482577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.483378 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.483724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.484102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.485103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.487068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecfe3d-7de5-42dd-b09e-db6cbc855a53 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.503663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 958/1903] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:32.504918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.517091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df7d79f-5c1b-4d9c-b31f-5878338c5a4e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:32.528490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:32.529187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.529466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.529737 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] PATH_INFO: `/v3/projects/11f875948df144b383d377da52dec79f/users/7c9ac76e01764b8f80431426d72298ee/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.558914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f875948df144b383d377da52dec79f, user_id=7c9ac76e01764b8f80431426d72298ee, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:32.562071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:32.719328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 946/1904] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:32 2022] PUT /identity/v3/projects/11f875948df144b383d377da52dec79f/users/7c9ac76e01764b8f80431426d72298ee/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:32.729759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:32.730039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:32.730325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:32.766737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] MFA Rules not processed for user `7c9ac76e01764b8f80431426d72298ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:32.768611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:32.824980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:32.879122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 959/1905] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:32.957137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:32.959716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:33.056425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f14b6-c351-4fc9-8b92-53b4ce88cc3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:33.060236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:33.060862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.063883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.063883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.073023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:33.074525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf24ea-e7ff-4bc1-830d-5492e4996e53 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:33.085233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 947/1906] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:32 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:33.103381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:33.191820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15ff7d-ba22-44e2-b14e-b927c0bf0e09 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:33.200643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:33.201640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.202019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.202513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] PATH_INFO: `/v3/users/7c9ac76e01764b8f80431426d72298ee` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.204107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9ac76e01764b8f80431426d72298ee)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:33.206335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:33.513536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9ac76e01764b8f80431426d72298ee'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:33.602802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9ac76e01764b8f80431426d72298ee'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:33.604121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9ac76e01764b8f80431426d72298ee'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:33.649392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9ac76e01764b8f80431426d72298ee'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:33.651286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9ac76e01764b8f80431426d72298ee'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:33.675207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 960/1907] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:33 2022] DELETE /identity/v3/users/7c9ac76e01764b8f80431426d72298ee => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:33.681768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.682869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.683432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.707906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:33.709499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:33.800818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 948/1908] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:33.812158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:33.814362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:33.895838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecef3d-e8c6-415a-ba8c-3a5c0727c608 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:33.908539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:33.909412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:33.909799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:33.910260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:33.927055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:33.927055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:33.932032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:33.946060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 961/1909] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:35.366100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:35.368440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:35.383836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:35.391201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:35.409015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b6afc-42bc-43d5-a0da-cdd99a3b0727 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:35.417285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26f150-459b-45ad-93ea-33317962cfaf service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:35.418744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:35.420014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:35.420786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:35.421902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] PATH_INFO: `/v3/projects/11f875948df144b383d377da52dec79f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:35.426837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f875948df144b383d377da52dec79f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:35.431132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:35.432231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:35.433150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:35.433572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:35.433938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] PATH_INFO: `/v3/users/1c2443818bd9452fa3010daf9193dab0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:35.437061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c2443818bd9452fa3010daf9193dab0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:35.439255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:35.619817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:35.762459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f875948df144b383d377da52dec79f', 'request_id': 'req-a3504b8f-7879-4a93-b9d9-751478b7f7e2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.769641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c2443818bd9452fa3010daf9193dab0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.773645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f875948df144b383d377da52dec79f', 'request_id': 'req-a3504b8f-7879-4a93-b9d9-751478b7f7e2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.930830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c2443818bd9452fa3010daf9193dab0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.931480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c2443818bd9452fa3010daf9193dab0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.950612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c2443818bd9452fa3010daf9193dab0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.951377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-133ed562-c0a9-4c30-80cc-15babc3ca7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c2443818bd9452fa3010daf9193dab0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:35.970046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 962/1910] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:35 2022] DELETE /identity/v3/users/1c2443818bd9452fa3010daf9193dab0 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:35.976092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff54175-99d3-43d5-9720-7c0cede7c4ae None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:35.976521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff54175-99d3-43d5-9720-7c0cede7c4ae None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:35.976902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff54175-99d3-43d5-9720-7c0cede7c4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.004670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-aff54175-99d3-43d5-9720-7c0cede7c4ae None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:36.006704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aff54175-99d3-43d5-9720-7c0cede7c4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:36.068969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f875948df144b383d377da52dec79f', 'request_id': 'req-a3504b8f-7879-4a93-b9d9-751478b7f7e2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:36.092883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] The token cache is being invalidate because project 11f875948df144b383d377da52dec79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:36.093195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:36.095909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3504b8f-7879-4a93-b9d9-751478b7f7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:36.102412 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 949/1911] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:35 2022] DELETE /identity/v3/projects/11f875948df144b383d377da52dec79f => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:36.114593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.114593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.114593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.163096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:36.164951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:36.168421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 963/1912] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:36.184477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 950/1913] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:36.193578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.195361 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.195654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.217577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.219366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:36.250711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:36.252449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:36.279984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 964/1914] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:36.288050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.317743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a2a7b-3a13-43da-bf44-0b11bf35ed95 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.334768 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:36.335445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.335721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.335987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.338644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:36.338961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:36.343105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:36.366393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd3864-6dfc-4e40-acea-eb6aac1db643 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.372063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:36.373205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.373740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.374163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.387508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:36.393027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:36.413669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 965/1915] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:36.413669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.422541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 951/1916] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:36.472606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00f35b-4038-4ef8-83f6-3e35584fe421 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.482229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:36.482863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.483166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.483488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.484390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:36.486324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:36.632677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 966/1917] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:36 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:36.638854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.653571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58674b-4a77-48c7-8332-6bdb52f48e45 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.795177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:36.796035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.796325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.796876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.797740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:36.807959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:36.884534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.886648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:36.907346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55976-501a-448c-839f-fc742d62816a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.930848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:36.930848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:36.930848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:36.930848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] PATH_INFO: `/v3/users/8585669077cd4b399bacbe364cc48fde` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:36.931331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8585669077cd4b399bacbe364cc48fde)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:36.932903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:36.944680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 952/1918] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:36 2022] POST /identity/v3/users => generated 287 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:36.945775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:36.980249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ee5e8-16b9-4079-8703-13b7040f9865 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:36.990635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:37.002204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:37.004525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.006294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.006809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.007792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:37.011951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:37.045559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 953/1919] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:37.047952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:37.076710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42890f6-fbb7-43c9-88b8-3d59d437527e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:37.089081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:37.091019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.091437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.091817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.092696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:37.094704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:37.121835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 954/1920] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:37.123003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:37.150992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675c56b-5b88-4d3d-96f4-9e2af1f20b03 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:37.159641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:37.160647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.161072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.161454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] PATH_INFO: `/v3/projects/37b64a5863a741fb899e2e1fa5fc3ebf/users/5f209a06c9d24fd0a0c44e0e531bf6a2/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.201370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b64a5863a741fb899e2e1fa5fc3ebf, user_id=5f209a06c9d24fd0a0c44e0e531bf6a2, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:37.201370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:37.203682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:37.260452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8585669077cd4b399bacbe364cc48fde'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.394212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 955/1921] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:37 2022] PUT /identity/v3/projects/37b64a5863a741fb899e2e1fa5fc3ebf/users/5f209a06c9d24fd0a0c44e0e531bf6a2/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:37.400819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:37.404269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:37.408312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8585669077cd4b399bacbe364cc48fde'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.409388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8585669077cd4b399bacbe364cc48fde'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.464267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8585669077cd4b399bacbe364cc48fde'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.469577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-90276ce3-2f11-4e58-ac81-0329a2edeaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8585669077cd4b399bacbe364cc48fde'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.508018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 967/1922] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:36 2022] DELETE /identity/v3/users/8585669077cd4b399bacbe364cc48fde => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:37.511917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7fb6-42b9-40b6-b66b-aea99b6b2e8f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.512201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7fb6-42b9-40b6-b66b-aea99b6b2e8f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.512651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7fb6-42b9-40b6-b66b-aea99b6b2e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.584913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f65e7fb6-42b9-40b6-b66b-aea99b6b2e8f None None] MFA Rules not processed for user `5f209a06c9d24fd0a0c44e0e531bf6a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:37.588398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f65e7fb6-42b9-40b6-b66b-aea99b6b2e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:37.612963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605fac9-7554-45f6-aa34-e29bc1199ad0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:37.698397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 968/1923] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:37.699837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.700121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.700401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.716294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:37.717020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.717397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.717977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] PATH_INFO: `/v3/projects/b63b9e25efb34fadab22a7d4466fac16` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.719329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63b9e25efb34fadab22a7d4466fac16)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:37.722379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:37.723296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:37.723746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:37.737783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 969/1924] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:37.737783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:37.741394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:37.753295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e2d30-5da2-471f-98d5-2a99f48fac19 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:37.760272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:37.760887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:37.761161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:37.761429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:37.762062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:37.764398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:37.937670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63b9e25efb34fadab22a7d4466fac16', 'request_id': 'req-0c533cad-b9c8-4486-bcb8-89c94ae73146'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.945438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63b9e25efb34fadab22a7d4466fac16', 'request_id': 'req-0c533cad-b9c8-4486-bcb8-89c94ae73146'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:37.967820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 970/1925] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:37 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:37.971951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:37.974029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.054174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63b9e25efb34fadab22a7d4466fac16', 'request_id': 'req-0c533cad-b9c8-4486-bcb8-89c94ae73146'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:38.073614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] The token cache is being invalidate because project b63b9e25efb34fadab22a7d4466fac16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:38.073905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:38.074693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:38.076139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a8035-350c-47f5-b2dd-1e5c3b849fa5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.076315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 956/1926] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:37 2022] DELETE /identity/v3/projects/b63b9e25efb34fadab22a7d4466fac16 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:38.083432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.083432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.090276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.091296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.091867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.092377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.095444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.111799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:38.157127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c533cad-b9c8-4486-bcb8-89c94ae73146 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.161827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.167836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.167836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.167836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.167836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.167836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.168292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.169321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde88b26-dc84-4f3d-90e1-033ddbae3258 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.183557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 971/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:38.185399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dde6-7e12-4aa9-a62f-757e4d3fd6e5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.185811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dde6-7e12-4aa9-a62f-757e4d3fd6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.186084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19dde6-7e12-4aa9-a62f-757e4d3fd6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.215755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1d19dde6-7e12-4aa9-a62f-757e4d3fd6e5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:38.217385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1d19dde6-7e12-4aa9-a62f-757e4d3fd6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.229541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 972/1928] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:38.237448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.238453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.238756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.268835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:38.271477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.286026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 973/1929] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:38.289821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.292046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 957/1930] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:38.293014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.300999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad603e78-9379-4602-be95-178a3e16b8db None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.305157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4802b8fc-3b11-4d87-b1dd-195fcfbe5020 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.306556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.307277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.308292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.308292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.308680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.310740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.311496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.312410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.312706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.313043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.313798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:38.314164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.316426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.320108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:38.325026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 974/1931] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:38.334260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.336191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 958/1932] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:38.340575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.350340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223e158-b4e6-4ed0-8229-3c1c39e3ef71 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.353881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdb043-e6eb-46d8-be7b-239f81382dc9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.355427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.356103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.356374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.356650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] PATH_INFO: `/v3/projects/be28e6904963459ab522d4e194b78038/users/216ce0eda85645afae6eeb3b86a1b44c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.360757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.361729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.362179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.362442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.363289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.365958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.376544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be28e6904963459ab522d4e194b78038, user_id=216ce0eda85645afae6eeb3b86a1b44c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.380048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.494752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 975/1933] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:38 2022] PUT /identity/v3/projects/be28e6904963459ab522d4e194b78038/users/216ce0eda85645afae6eeb3b86a1b44c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:38.496856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.532472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 959/1934] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/projects => generated 371 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:38.536944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.585944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a7c8f-162f-4d02-81c6-d1509b8bfa78 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.639404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9d3f0-4f6d-45ec-905b-5a19060bd24e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.645582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.646596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.647427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.647939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.649106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.652084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.665210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.665839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.666552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.666833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.667524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.669143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.682515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 976/1935] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:38.686799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.701990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f28ee-6243-4dcc-a63a-fd82530fbecb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.709683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.710503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.711176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.711176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] PATH_INFO: `/v3/projects/be28e6904963459ab522d4e194b78038/users/216ce0eda85645afae6eeb3b86a1b44c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.713542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be28e6904963459ab522d4e194b78038, user_id=216ce0eda85645afae6eeb3b86a1b44c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.716401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7dc30-aba8-40eb-9569-38d904abcf01 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.731835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 977/1936] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:38 2022] PUT /identity/v3/projects/be28e6904963459ab522d4e194b78038/users/216ce0eda85645afae6eeb3b86a1b44c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:38.743257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.743588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.744807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.761127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 960/1937] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:38.762113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.775062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091beb4-b492-4c9d-a19d-1b9aff77c5dc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.790236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] MFA Rules not processed for user `216ce0eda85645afae6eeb3b86a1b44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:38.791713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.846595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.847611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.847886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.848157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.849303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.851954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.869421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 961/1938] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:38.883709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.883709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:38.884595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 978/1939] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:38.890503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.893862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3cde6-746e-4558-9933-cc3e825796bf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.901979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.903462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.903856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.904031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] PATH_INFO: `/v3/projects/5c2379c7f0414c06a32686c44c8f4e39` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.906957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c2379c7f0414c06a32686c44c8f4e39)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.909700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.910275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb6569-ecd2-4358-9d0d-60ca6ae2306a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:38.916980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:38.918479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:38.918997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:38.919417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:38.924647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:38.924647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:38.939332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 979/1940] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:38.940680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:38.945989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.014765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8babfb-4c7a-48e2-98f2-b58241a3c813 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:39.019134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:39.019778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:39.020220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:39.020504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:39.022812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.064504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:39.077759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c2379c7f0414c06a32686c44c8f4e39', 'request_id': 'req-447a60f1-1387-47e6-8dad-8eabfb5cb652'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.082704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c2379c7f0414c06a32686c44c8f4e39', 'request_id': 'req-447a60f1-1387-47e6-8dad-8eabfb5cb652'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.105431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:39.107126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:39.117878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 980/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:39.120441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:39.134911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e321f4-ff2c-40b6-a83f-469d126e08ac service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:39.175564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c2379c7f0414c06a32686c44c8f4e39', 'request_id': 'req-447a60f1-1387-47e6-8dad-8eabfb5cb652'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.197891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] The token cache is being invalidate because project 5c2379c7f0414c06a32686c44c8f4e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:39.198467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.199295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.201081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 962/1942] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:38 2022] DELETE /identity/v3/projects/5c2379c7f0414c06a32686c44c8f4e39 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:39.205649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:39.212289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.213310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:39.214517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:39.215259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:39.218011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] PATH_INFO: `/v3/projects/c666a18b5b454c16b0a357ecf11f0146/users/b08f847ded484114bd3897cf07a48f1d/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:39.229948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:39.252514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a60f1-1387-47e6-8dad-8eabfb5cb652 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:39.266089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c666a18b5b454c16b0a357ecf11f0146, user_id=b08f847ded484114bd3897cf07a48f1d, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:39.268856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50417b59-bc90-4360-90a6-90695c15da92 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:39.270423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:39.271508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:39.271771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:39.272052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] PATH_INFO: `/v3/users/92c71830aad44368b4e47733a115f8ab` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:39.273814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c71830aad44368b4e47733a115f8ab)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:39.277314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:39.352138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 981/1943] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:39 2022] PUT /identity/v3/projects/c666a18b5b454c16b0a357ecf11f0146/users/b08f847ded484114bd3897cf07a48f1d/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:39.363013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:39.363772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:39.363772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:39.392576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] MFA Rules not processed for user `b08f847ded484114bd3897cf07a48f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:39.393824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.483234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 982/1944] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:39.493912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:39.495654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.556179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c71830aad44368b4e47733a115f8ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.557869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c71830aad44368b4e47733a115f8ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.582023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a98cf-0cd2-4e78-8679-e51baa977d75 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:39.586908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:39.587644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:39.587928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:39.588239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:39.590237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:39.606480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:39.608002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:39.621272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 983/1945] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:39.688234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c71830aad44368b4e47733a115f8ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.702123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c71830aad44368b4e47733a115f8ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.719926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4353c04c-d3fe-41d8-b130-c00a791d8ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c71830aad44368b4e47733a115f8ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:39.732720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 963/1946] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:39 2022] DELETE /identity/v3/users/92c71830aad44368b4e47733a115f8ab => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:39.965367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:40.550510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:40.550510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:40.646412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbaa7c-8d21-4177-a27c-cc097e7150c2 service glance] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:40.708490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:40.709175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:40.709475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:40.709745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] PATH_INFO: `/v3/projects/4b09382991a144769cbfa01387a7cd17` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:40.710967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b09382991a144769cbfa01387a7cd17)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:40.712924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:40.831603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b09382991a144769cbfa01387a7cd17', 'request_id': 'req-695d9c74-c82e-43dc-a76a-858d78f0728c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:40.841612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b09382991a144769cbfa01387a7cd17', 'request_id': 'req-695d9c74-c82e-43dc-a76a-858d78f0728c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:40.928949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b09382991a144769cbfa01387a7cd17', 'request_id': 'req-695d9c74-c82e-43dc-a76a-858d78f0728c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:40.941161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] The token cache is being invalidate because project 4b09382991a144769cbfa01387a7cd17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:40.941478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:40.942212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-695d9c74-c82e-43dc-a76a-858d78f0728c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:40.943977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 984/1947] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:40 2022] DELETE /identity/v3/projects/4b09382991a144769cbfa01387a7cd17 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:40.948961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:40.949531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:40.955819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:40.971196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:40.972482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.054156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:41.058837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 964/1948] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.067394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.067673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.067991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.088919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:41.090203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.100392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 985/1949] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.104039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.115200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d23fa8-520b-4d72-98b7-baf26c37c583 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.119854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.120465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.120762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.121031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.121682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:41.121978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.124342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.127027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:41.138965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 965/1950] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:41.143686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.154141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8c984-3479-49a6-ba1c-a5b0318377d0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.160200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.160815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.161088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.161359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.162792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.164829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.294873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 986/1951] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:41 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:41.299703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.310435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c2a8e-5834-450b-8fa0-4aaafdd8bc6a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.371830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.372348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.372631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.372898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.373539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.375496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.501021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 966/1952] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:41 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:41.502310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.513785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9745622-c1f6-46bb-8f6d-fa3ef2bba566 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.519968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.520737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.521065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.524180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.524180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.524180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.536210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 987/1953] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:41.541721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.556874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d580941-0a51-4833-8b87-906ff22346fb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.562200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.562831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.563057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.563640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.566879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.568664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a5dc9-4bb9-4c5a-b72c-6e32e791b488 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.584180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 967/1954] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:41.587415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.597776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ac0f9-32bb-468f-935a-e4ea70c02b39 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.603997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.604700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.604981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.605259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] PATH_INFO: `/v3/projects/4a999d0bc4f64c54ad1b4667e4dee722/users/c496969a314a4e4991c92af090a36417/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.620962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a999d0bc4f64c54ad1b4667e4dee722, user_id=c496969a314a4e4991c92af090a36417, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.623747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.765472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 988/1955] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:41 2022] PUT /identity/v3/projects/4a999d0bc4f64c54ad1b4667e4dee722/users/c496969a314a4e4991c92af090a36417/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:41.777906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.778224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.778567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.807504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] MFA Rules not processed for user `c496969a314a4e4991c92af090a36417`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:41.808858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.879945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 968/1956] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:41.892126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:41.894044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:41.910774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:41.961832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5462bdb-c871-40d4-95e4-786ced3791e3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:41.966437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:41.967076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:41.967546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:41.967660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:41.978437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:41.979945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ed095-1c7b-46d9-842a-8dc11d9daaf9 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:41.989796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 989/1957] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:42.019774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:42.092493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b49c0-4bd2-470e-a2b6-569199268184 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:42.097499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:42.098231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.098547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.098863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] PATH_INFO: `/v3/users/c496969a314a4e4991c92af090a36417` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.101191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c496969a314a4e4991c92af090a36417)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:42.103568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:42.108813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:42.360265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c496969a314a4e4991c92af090a36417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.361020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c496969a314a4e4991c92af090a36417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.453352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c496969a314a4e4991c92af090a36417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.462572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c496969a314a4e4991c92af090a36417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.482068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c496969a314a4e4991c92af090a36417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:42.489941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 969/1958] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:42 2022] DELETE /identity/v3/users/c496969a314a4e4991c92af090a36417 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:42.495580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.496417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.496417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.518942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:42.520145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:42.583518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 990/1959] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:42.594203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:42.595704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:42.661750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8877f60-6261-4d36-8505-8439365f3cce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:42.667100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:42.667752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:42.668054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:42.668321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:42.679570 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:42.681215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a4e220-ab92-448a-abe4-341779a2552e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:42.696453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 970/1960] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:43.273827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.274820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:43.287901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e4677-1cb5-473d-815b-adf0b7f5444c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.292951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.293629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.293926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.294178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] PATH_INFO: `/v3/projects/4a999d0bc4f64c54ad1b4667e4dee722` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.295491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a999d0bc4f64c54ad1b4667e4dee722)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.297429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.466428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a999d0bc4f64c54ad1b4667e4dee722', 'request_id': 'req-a7c14ec2-6769-4330-81c9-48aa5bdde78c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.473028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a999d0bc4f64c54ad1b4667e4dee722', 'request_id': 'req-a7c14ec2-6769-4330-81c9-48aa5bdde78c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.593695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a999d0bc4f64c54ad1b4667e4dee722', 'request_id': 'req-a7c14ec2-6769-4330-81c9-48aa5bdde78c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.606476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] The token cache is being invalidate because project 4a999d0bc4f64c54ad1b4667e4dee722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:43.606763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.607488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a7c14ec2-6769-4330-81c9-48aa5bdde78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:43.609070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 991/1961] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:43 2022] DELETE /identity/v3/projects/4a999d0bc4f64c54ad1b4667e4dee722 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:43.614160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.614676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.614765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.637065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:43.638289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:43.709079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 971/1962] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:43.717863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.718151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.718428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.740628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:43.742006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:43.750674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 992/1963] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:43.754613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.754744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.756159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:43.764722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00af873-0896-463e-aa46-c01a59a876bf None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.769252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ad1db-cf37-4230-9172-700ef1c89d8f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.769901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.770526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.770812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.771096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.782013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:43.782013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.782013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.782390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.782390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.782390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.782390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] PATH_INFO: `/v3/users/5f209a06c9d24fd0a0c44e0e531bf6a2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.782851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f209a06c9d24fd0a0c44e0e531bf6a2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.782851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.791197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 993/1964] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:34:43.792431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:43.802944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ed5a7-7cd0-4883-bd12-a035776128ba admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:43.803197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:43.808547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:43.809151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:43.809435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:43.809714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:43.810362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:43.812334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:43.829779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:43.941898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 994/1965] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:43 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:43.946654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.022214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e0986-30b4-4f5b-b137-4ca7ac9fa13a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.032232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f209a06c9d24fd0a0c44e0e531bf6a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:44.033291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f209a06c9d24fd0a0c44e0e531bf6a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:44.034260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.034969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.035533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.035811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.036481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.039134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.136785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f209a06c9d24fd0a0c44e0e531bf6a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:44.149306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f209a06c9d24fd0a0c44e0e531bf6a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:44.156458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-606bf63b-73ee-49de-ae01-bc95e9c465fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f209a06c9d24fd0a0c44e0e531bf6a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:44.162697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 972/1966] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:43 2022] DELETE /identity/v3/users/5f209a06c9d24fd0a0c44e0e531bf6a2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:44.167932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.168219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.168491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.178414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 995/1967] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:43 2022] POST /identity/v3/users => generated 289 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:44.187195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.188532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:44.188953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.189816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.203017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25e00b-e55e-4373-9ce0-4f023999579f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.260506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 973/1968] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:44.270886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.273656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.290173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.290885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.291196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.291479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.292228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.293862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.308835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 996/1969] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:44.311462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.324511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90bf8-eb21-4e26-a7e9-1c2abae8f9bb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.329632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.330429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.330905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.331373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.332031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.333693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.346335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 997/1970] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:44.347953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0bfd4e-3065-451a-a7c4-9076211b3042 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.351496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.354572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.355249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.355601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.355877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.362101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33962a-dd4b-4380-9398-781d9fe773c8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.367550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.367854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.368215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.368495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.368792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] PATH_INFO: `/v3/projects/66808b6f25e74338a17b1ac03568290c/users/4ae85b5e67574e1c9ffaf6b6445ff414/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.369538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5030-1dbc-4423-90e2-281b4a478a15 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.380299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 974/1971] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:44.386690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=66808b6f25e74338a17b1ac03568290c, user_id=4ae85b5e67574e1c9ffaf6b6445ff414, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.389781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.475316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 998/1972] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:44 2022] PUT /identity/v3/projects/66808b6f25e74338a17b1ac03568290c/users/4ae85b5e67574e1c9ffaf6b6445ff414/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:44.485825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.486113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.486405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.520746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] MFA Rules not processed for user `4ae85b5e67574e1c9ffaf6b6445ff414`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:44.522053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.554417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:44.592389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 975/1973] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:44.606499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:44.612926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:44.694092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:44.702074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c56c3-e924-4a42-984a-be31fcd51c3b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:44.707042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:44.707713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:44.707950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:44.708233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:44.718322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:44.719826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:44.729977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 999/1974] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:45.356828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.358358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:45.446316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23b2ba-26b1-4b76-a0cc-065f05e70b82 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.450781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.451459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.451737 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.452012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] PATH_INFO: `/v3/projects/37b64a5863a741fb899e2e1fa5fc3ebf` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.453181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b64a5863a741fb899e2e1fa5fc3ebf)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.455124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:45.624297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b64a5863a741fb899e2e1fa5fc3ebf', 'request_id': 'req-9a426683-663c-4a8a-9cf9-c92cd86090f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:45.631208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b64a5863a741fb899e2e1fa5fc3ebf', 'request_id': 'req-9a426683-663c-4a8a-9cf9-c92cd86090f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:45.712936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b64a5863a741fb899e2e1fa5fc3ebf', 'request_id': 'req-9a426683-663c-4a8a-9cf9-c92cd86090f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:45.724400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] The token cache is being invalidate because project 37b64a5863a741fb899e2e1fa5fc3ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:45.724688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:45.725336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:45.728461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 976/1975] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:45 2022] DELETE /identity/v3/projects/37b64a5863a741fb899e2e1fa5fc3ebf => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:45.732043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:45.748700 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3283c-4e2f-47eb-b2ce-06b093eb6c2d service glance] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:45.820464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:45.821144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:45.821444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:45.821713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] PATH_INFO: `/v3/users/216ce0eda85645afae6eeb3b86a1b44c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:45.822830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216ce0eda85645afae6eeb3b86a1b44c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:45.824906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:46.205259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216ce0eda85645afae6eeb3b86a1b44c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:46.313391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216ce0eda85645afae6eeb3b86a1b44c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:46.314227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216ce0eda85645afae6eeb3b86a1b44c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:46.332407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216ce0eda85645afae6eeb3b86a1b44c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:46.333220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-aaad51d6-19a9-4642-b728-6c830573d733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216ce0eda85645afae6eeb3b86a1b44c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:46.354722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1000/1976] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:45 2022] DELETE /identity/v3/users/216ce0eda85645afae6eeb3b86a1b44c => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:46.871922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:46.871922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:46.890317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a426683-663c-4a8a-9cf9-c92cd86090f4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:46.958475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:46.959129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:46.959437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:46.959706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] PATH_INFO: `/v3/projects/be28e6904963459ab522d4e194b78038` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:46.961798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=be28e6904963459ab522d4e194b78038)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:46.963814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:46.966479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:47.145267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be28e6904963459ab522d4e194b78038', 'request_id': 'req-f201474e-457f-4b3a-88d8-98a5003108bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:47.152496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be28e6904963459ab522d4e194b78038', 'request_id': 'req-f201474e-457f-4b3a-88d8-98a5003108bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:47.245449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be28e6904963459ab522d4e194b78038', 'request_id': 'req-f201474e-457f-4b3a-88d8-98a5003108bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:47.267722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] The token cache is being invalidate because project be28e6904963459ab522d4e194b78038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:47.270739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:47.270739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f201474e-457f-4b3a-88d8-98a5003108bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:47.270739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 977/1977] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:46 2022] DELETE /identity/v3/projects/be28e6904963459ab522d4e194b78038 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:47.276832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.277338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.277437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.302373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:47.303801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:47.371892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1001/1978] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:34:47.379867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.380154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.380431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.401541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:47.402752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:47.410936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 978/1979] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:47.415502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:47.425583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cb87f-5a03-4f18-88ac-1f4c01a60fa5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:47.430782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:47.431454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.431732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.431998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.432698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:47.432998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:47.434617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:47.437512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:47.447784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1002/1980] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:47.452105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:47.464814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2528a1f-ab8c-4ff0-902c-9e7ce91fd74c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:47.473001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:47.473947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.474386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.474840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.477117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:47.480626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:47.626819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 979/1981] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:47 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:47.632206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:47.644686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140eb1bc-8141-4504-960e-127a74bef436 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:47.710107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:47.711034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.711400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.711673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.712416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:47.714296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:47.930068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1003/1982] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:47 2022] POST /identity/v3/users => generated 292 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:34:47.935591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:47.955476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be817063-4aa2-4a91-8776-d856934a82ba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:47.963562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:47.964688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:47.965193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:47.965714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:47.966915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:47.969928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:47.983270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 980/1983] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:47.988288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:48.000723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5772bb-2878-4580-af08-0fcd5609f78f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:48.005466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:48.006066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.006396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.006670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.007478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:48.009076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fab44-902a-424b-9a34-d9395f2c6faa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:48.019924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1004/1984] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:34:48.024179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:48.034617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ffbff-99dd-486e-94ea-d448b4730f12 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:48.040240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:48.040915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.041233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.041522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] PATH_INFO: `/v3/projects/e180e041b75e4c7a9143750ade1cbd1c/users/4d913f59b9024e90b554f298abf38519/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.059250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e180e041b75e4c7a9143750ade1cbd1c, user_id=4d913f59b9024e90b554f298abf38519, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:48.061904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:48.186569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 981/1985] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:48 2022] PUT /identity/v3/projects/e180e041b75e4c7a9143750ade1cbd1c/users/4d913f59b9024e90b554f298abf38519/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:48.198402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.198804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.199083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.231282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] MFA Rules not processed for user `4d913f59b9024e90b554f298abf38519`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:48.232553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:48.316808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1005/1986] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:48.329670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:48.331641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:48.404438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d389f-a2b3-477f-8d54-c46d94ecd000 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:48.408329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:48.408995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:48.409272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:48.409544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:48.418652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:48.420128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9770-671e-4666-a1f6-ed84d250241c service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:48.429849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 982/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:52.792791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:52.874503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2ab6-a17c-4f07-9073-cadbef1f0e5f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:52.883493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:52.885231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:52.885550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:52.885988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] PATH_INFO: `/v3/users/4d913f59b9024e90b554f298abf38519` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:52.887340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d913f59b9024e90b554f298abf38519)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:52.889553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:52.898611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.147687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d913f59b9024e90b554f298abf38519'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.254366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d913f59b9024e90b554f298abf38519'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.255020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d913f59b9024e90b554f298abf38519'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.272810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d913f59b9024e90b554f298abf38519'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.273716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d913f59b9024e90b554f298abf38519'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:53.289356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1006/1988] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:52 2022] DELETE /identity/v3/users/4d913f59b9024e90b554f298abf38519 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:34:53.294549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.295084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.295084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.315364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:53.316850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.354509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:53.390987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 983/1989] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:53.404019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:53.406174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:53.482550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf98cd-cafc-460b-b65e-ed728e16cf46 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:53.486178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:53.486778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:53.487046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:53.487353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:53.497409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:53.498842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3d0e7-8834-4125-b62a-0b0063956966 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:53.508552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1007/1990] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:54.073678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.078188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.088339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2d203-9cab-4fea-ac07-d81db919e9a6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.094400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.095199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.095555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.095898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] PATH_INFO: `/v3/projects/e180e041b75e4c7a9143750ade1cbd1c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.097184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e180e041b75e4c7a9143750ade1cbd1c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.099189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.257111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e180e041b75e4c7a9143750ade1cbd1c', 'request_id': 'req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:54.264674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e180e041b75e4c7a9143750ade1cbd1c', 'request_id': 'req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:54.376831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e180e041b75e4c7a9143750ade1cbd1c', 'request_id': 'req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:54.394536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] The token cache is being invalidate because project e180e041b75e4c7a9143750ade1cbd1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:34:54.395040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:54.395862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1da9ff6e-fcae-4fb0-983c-eef6f3c33a64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:34:54.397607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 984/1991] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:34:54 2022] DELETE /identity/v3/projects/e180e041b75e4c7a9143750ade1cbd1c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:54.404093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.404657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.405105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.428529 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:54.429862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.518779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1008/1992] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:54.527367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.527924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.528391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.551186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:54.552435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:54.562212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 985/1993] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:54.566879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.578582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cffd96-9ff4-4618-ae9d-03b4a6db4f57 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.583590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.584234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.584511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.584781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.585435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:34:54.585743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.588101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.605854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1009/1994] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:34:54.609922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.620353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67025a9-1635-45d0-a44c-37ca19abef5a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.626007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.626638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.626917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.627301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.627948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.629987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.779303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 986/1995] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.785102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.798697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cea410c-2b0a-49dd-b9b6-37b726b056ee admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:54.813308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:54.881997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:54.882572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:54.883044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:54.883358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:54.885562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:54.886899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:54.978171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1010/1996] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:34:54 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:34:54.982882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:54.998472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c584a-d9b2-48fa-ab9a-bd9d2060b77e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.004222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.005085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.005701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.006000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.006841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.010031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.022880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 987/1997] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:55.027813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.038265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253aa3a0-8b17-40bc-8503-d7f748dbd6ee admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.043683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.044323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.044600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.044868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.045513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.047190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b4f59-530a-4a6a-b241-d4303fa17119 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.061693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1011/1998] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:34:55.068244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.080426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8d4f0-633e-4c94-818b-5006f1b8485d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.085389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.086049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.086327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.086595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] PATH_INFO: `/v3/projects/22dcb2335f19495e91b8de154d2110ac/users/a8a6a1293fe44530a07d8bc4b0f74613/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.106287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dcb2335f19495e91b8de154d2110ac, user_id=a8a6a1293fe44530a07d8bc4b0f74613, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.109072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.265951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 988/1999] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:34:55 2022] PUT /identity/v3/projects/22dcb2335f19495e91b8de154d2110ac/users/a8a6a1293fe44530a07d8bc4b0f74613/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:34:55.269029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c1e7-5b71-41f3-b593-90abc77e302a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.269379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c1e7-5b71-41f3-b593-90abc77e302a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.269659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18c1e7-5b71-41f3-b593-90abc77e302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.302103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2e18c1e7-5b71-41f3-b593-90abc77e302a None None] MFA Rules not processed for user `a8a6a1293fe44530a07d8bc4b0f74613`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:55.303358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2e18c1e7-5b71-41f3-b593-90abc77e302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.382117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1012/2000] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:55.391932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:55.392514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:55.456286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d26a42-efc9-4583-849b-19703fdbfeb5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:55.460112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:55.460755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:55.461035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:55.461306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:55.467775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:55.472123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:55.473610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5958c8-b33a-4f8e-a4c8-074bb9231154 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:55.485173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 989/2001] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:34:58.458258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0287d5-addb-4569-b9f8-b5862fa3d12e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:58.458758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0287d5-addb-4569-b9f8-b5862fa3d12e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:58.458867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0287d5-addb-4569-b9f8-b5862fa3d12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:58.491268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ff0287d5-addb-4569-b9f8-b5862fa3d12e None None] MFA Rules not processed for user `a8a6a1293fe44530a07d8bc4b0f74613`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:58.493140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ff0287d5-addb-4569-b9f8-b5862fa3d12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:58.504605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1013/2002] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:34:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:59.631415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ee0a6-3f46-49a5-9c62-1258a574dd0f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.638702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ee0a6-3f46-49a5-9c62-1258a574dd0f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.638702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ee0a6-3f46-49a5-9c62-1258a574dd0f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.638702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 990/2003] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 01 16:34:59.648850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d358f-a160-4dc1-ab57-9ef5e8cc84b4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.650305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d358f-a160-4dc1-ab57-9ef5e8cc84b4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.651320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d358f-a160-4dc1-ab57-9ef5e8cc84b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.682143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a28d358f-a160-4dc1-ab57-9ef5e8cc84b4 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:59.684311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a28d358f-a160-4dc1-ab57-9ef5e8cc84b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:59.769131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1014/2004] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:34:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:59.783261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.783664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.784032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.785731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 991/2005] 10.222.0.16 () {54 vars in 961 bytes} [Thu Sep 1 16:34:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:34:59.792787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.793491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.793872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.816764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:59.819753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:59.828599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:34:59.831109 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:34:59.856462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1015/2006] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Sep 1 16:34:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:34:59.865138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:34:59.877335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:34:59.885244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c4646-c2ee-415a-84cf-68843b5fefb6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:59.896663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:59.897822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.898366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.898901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.918257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:34:59.921222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:34:59.940551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1016/2007] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:34:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:34:59.990510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55141b9-8b2f-44a6-96a0-fec7b141afd0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:34:59.995918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:34:59.996588 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:34:59.996868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:34:59.997139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] PATH_INFO: `/v3/users/662f85837db7411e8695d971413fc5ba` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:34:59.998391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=662f85837db7411e8695d971413fc5ba)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:00.000449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:00.206393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662f85837db7411e8695d971413fc5ba'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.207070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662f85837db7411e8695d971413fc5ba'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.271328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662f85837db7411e8695d971413fc5ba'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.280062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662f85837db7411e8695d971413fc5ba'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.293073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-888e8859-8259-46eb-81b1-39c0bb425099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662f85837db7411e8695d971413fc5ba'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.299071 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 992/2008] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:34:59 2022] DELETE /identity/v3/users/662f85837db7411e8695d971413fc5ba => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:00.580515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:00.594496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784e3a6-6f29-4a86-abb1-7d0462f3a741 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:00.665962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:00.666745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:00.666862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:00.667130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] PATH_INFO: `/v3/projects/c504a04e6b84427b9732157524a4aeef` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:00.668322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c504a04e6b84427b9732157524a4aeef)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:00.670242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:00.795737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c504a04e6b84427b9732157524a4aeef', 'request_id': 'req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.803883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c504a04e6b84427b9732157524a4aeef', 'request_id': 'req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.879894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c504a04e6b84427b9732157524a4aeef', 'request_id': 'req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.895643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] The token cache is being invalidate because project c504a04e6b84427b9732157524a4aeef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:00.896200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.897218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93d5d544-8d72-4f4a-bb89-0d2f13e395fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:00.899010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1017/2009] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:00 2022] DELETE /identity/v3/projects/c504a04e6b84427b9732157524a4aeef => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:00.905056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:00.905709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:00.905807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:00.929802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:00.931927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.001804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 993/2010] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:01.010724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.011012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.011320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.032543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:01.033775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.043559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1018/2011] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:01.048025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.059840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91b7a42-01c7-4cee-aadc-afa741570d42 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.065223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.066342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.066553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.066725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.067376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:01.067682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.069403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.104010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:01.107792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 994/2012] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:01.109326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.126069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d3a0d-f22f-440c-a880-2d1d0a94c047 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.133967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.135023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.135533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.135974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.137054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.140628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.284729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1019/2013] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:01 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:01.289044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.299845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd868a8-2000-4def-9140-271777cf7b50 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.358271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.358803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.359086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.359391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.360033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.361989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.460438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 995/2014] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:01 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:01.466018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.476739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a629116-f1fc-4f3b-9f8f-1b162551b9d3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.481693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.482300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.482579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.482849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.483652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.485251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.497381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1020/2015] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:01.501716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.514762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd4767-2998-47a2-93fd-0d453c1ef1fc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.519496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.520100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.520378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.520652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.521301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.522908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43443782-6060-432c-a829-b2fa945eef28 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.534599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 996/2016] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:01.539638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.550412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fea1c4-7b37-49b3-aa10-c701ebcd95b5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.555451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.556429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.556836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.557112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] PATH_INFO: `/v3/projects/af19a8ade9ac443a84bc73c65fe8b50c/users/0393d4a271974edcbf36e003396e8d47/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.575504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af19a8ade9ac443a84bc73c65fe8b50c, user_id=0393d4a271974edcbf36e003396e8d47, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.578290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.640399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1021/2017] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:01 2022] PUT /identity/v3/projects/af19a8ade9ac443a84bc73c65fe8b50c/users/0393d4a271974edcbf36e003396e8d47/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:01.651743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.652032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.652308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.688653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] MFA Rules not processed for user `0393d4a271974edcbf36e003396e8d47`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:01.689871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.777343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 997/2018] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:01.788540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:01.790795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:01.815834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:01.878686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e6693-05ec-48dc-9642-9e9db5b01320 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:01.885643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:01.886288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:01.886568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:01.886840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:01.897235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:01.898775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6071e1-6c01-4aac-b426-91e0ad9504df service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:01.909551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1022/2019] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:06.654161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:06.780097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003844ae-91f7-42a5-880d-1afd313a18b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:06.786033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:06.786983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:06.787604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:06.788056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] PATH_INFO: `/v3/users/0393d4a271974edcbf36e003396e8d47` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:06.789951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0393d4a271974edcbf36e003396e8d47)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:06.796529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:06.811374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:07.139079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0393d4a271974edcbf36e003396e8d47'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.139799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0393d4a271974edcbf36e003396e8d47'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.256906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0393d4a271974edcbf36e003396e8d47'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.267052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0393d4a271974edcbf36e003396e8d47'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.279848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0393d4a271974edcbf36e003396e8d47'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:07.286385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 998/2020] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:06 2022] DELETE /identity/v3/users/0393d4a271974edcbf36e003396e8d47 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:07.292454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.293264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.293429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.331634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:07.334005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.413563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1023/2021] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:07.424245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:07.425707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:07.495552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912884-cf16-4727-ac2d-a70f5d49fd54 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:07.500551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:07.501146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:07.501584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:07.501966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:07.516324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:07.517923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa36162-c977-4a42-ac1a-8df92ba04d71 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:07.530810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 999/2022] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:08.267862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.269383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.282603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98747e66-0694-449b-8883-e8c03a5fc0e8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.291463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.292450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.292570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.293124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] PATH_INFO: `/v3/projects/af19a8ade9ac443a84bc73c65fe8b50c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.298833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] RBAC: Authorizing `identity:delete_project(project_id=af19a8ade9ac443a84bc73c65fe8b50c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.298833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.481780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af19a8ade9ac443a84bc73c65fe8b50c', 'request_id': 'req-95378bfc-067f-4d49-9911-ac5e5860febd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.489198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af19a8ade9ac443a84bc73c65fe8b50c', 'request_id': 'req-95378bfc-067f-4d49-9911-ac5e5860febd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.598302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af19a8ade9ac443a84bc73c65fe8b50c', 'request_id': 'req-95378bfc-067f-4d49-9911-ac5e5860febd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.611544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] The token cache is being invalidate because project af19a8ade9ac443a84bc73c65fe8b50c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:08.611833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.612516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95378bfc-067f-4d49-9911-ac5e5860febd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:08.614022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1024/2023] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:08 2022] DELETE /identity/v3/projects/af19a8ade9ac443a84bc73c65fe8b50c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:08.619967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.620595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.620595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.642732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.644287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.715349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1000/2024] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:08.723479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.723653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.723923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.745869 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:08.747099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:08.755769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1025/2025] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:08.760621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.773700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042de26-a1b6-4f84-a8ad-2598d6762661 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.795266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.795266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.795266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.795266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.796363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:08.796363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.796363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.810218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1001/2026] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:08.814692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:08.824854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee19869-1abd-4e8e-9fe3-8f43f8daf5dc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:08.829971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:08.830703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:08.831049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:08.831779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:08.832179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:08.834169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:08.992902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:09.009926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1026/2027] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:08 2022] POST /identity/v3/projects => generated 393 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:09.013097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.026882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c317d5d-8498-45e0-978c-187abe50923f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.050170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:09.135525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.136582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.137076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.137561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.139611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.142328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.253206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1002/2028] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:09.257502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.272986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b3a0a-ebc6-42e5-92c7-35c4c3cdd3dc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.279705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.280314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.280589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.280909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.281698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.283608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.294588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1027/2029] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:09.299227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.315923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666b8df-46af-49d6-b24a-fe42e5d41330 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.315923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.316885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.317165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.317496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.318092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.319761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0441450-6701-497e-9693-6ef0fca31b1f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.335195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1003/2030] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:09.338448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.357573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947847d-3109-4d77-9f8f-bc6e57ff0446 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.364866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.366574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.366915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.367377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] PATH_INFO: `/v3/projects/8f2962b0b7ef466c81c3db0d6dfc23a9/users/4ca18e731aca48199461f07b2b92aca0/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.391420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2962b0b7ef466c81c3db0d6dfc23a9, user_id=4ca18e731aca48199461f07b2b92aca0, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.394329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.534928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1028/2031] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:09 2022] PUT /identity/v3/projects/8f2962b0b7ef466c81c3db0d6dfc23a9/users/4ca18e731aca48199461f07b2b92aca0/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:09.545873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.546158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.546433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.578057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] MFA Rules not processed for user `4ca18e731aca48199461f07b2b92aca0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:09.579077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:09.654394 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1004/2032] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:09.659141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.733044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de53a09d-32e6-4fc0-8ed5-06cdf51ba51d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.739133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.753273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.753273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.753273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.753273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.753273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:09.902242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1029/2033] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:09.902658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:09.914408 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850d17c-1cb0-4485-92dd-815e1d4223b8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:09.984849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:09.985468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:09.985745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:09.986016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:09.986657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:09.988675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.233758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1005/2034] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:09 2022] POST /identity/v3/users => generated 302 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:10.237920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.253712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df9459-a9d5-4d2a-a62d-6c9d2e690179 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.264100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.264734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.265015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.265295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.265964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.267675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.293620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1030/2035] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:10.299953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.323236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3972b-8723-455b-b6cc-abb58e604f4c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.324360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.326929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.327803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.328192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] PATH_INFO: `/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce/users/185ad21808c34e37a67729e14a76adde/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.346731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:10.355876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf31e1215b642648a5fbfef4e85c4ce, user_id=185ad21808c34e37a67729e14a76adde, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.359231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.436902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1006/2036] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:10 2022] PUT /identity/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce/users/185ad21808c34e37a67729e14a76adde/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:10.443623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.465779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36327e68-d086-462f-951b-b18abd4f1d0f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.587784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.589060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.589553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.590035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.591218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.594362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ad8e-8ac1-4196-9027-085a4d5f50b0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.620067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1031/2037] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:10.620359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.630929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923568-0c95-4ee9-8ba5-772298e56084 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.636200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.636874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.637151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.637419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] PATH_INFO: `/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce/users/185ad21808c34e37a67729e14a76adde/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.640005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf31e1215b642648a5fbfef4e85c4ce, user_id=185ad21808c34e37a67729e14a76adde, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.642719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.658845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1007/2038] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:10 2022] PUT /identity/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce/users/185ad21808c34e37a67729e14a76adde/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:10.673375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.673660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.673935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.720476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] MFA Rules not processed for user `185ad21808c34e37a67729e14a76adde`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:10.723404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.758222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:10.823912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1032/2039] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:10.836183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.839089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.904906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1ffc3-5dea-4ba1-a40a-9f9368dfc26a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:10.909105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:10.909731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:10.910002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:10.910276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:10.921263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:10.922729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:10.933660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1008/2040] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:10.983681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:10.985124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:10.997454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35797a9-a789-4815-a808-8edc1e0eac93 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:11.001432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:11.001775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:11.002044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:11.002307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:11.081117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:11.082690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e2a16-4efe-4592-b6e2-0e78842cd894 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:11.094944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1033/2041] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:18.640816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:18.731832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93bfcd-3015-4fd0-badb-df066098638b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:18.737117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:18.738372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:18.738372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:18.738694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] PATH_INFO: `/v3/users/4ca18e731aca48199461f07b2b92aca0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:18.739905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca18e731aca48199461f07b2b92aca0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:18.741883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:18.952595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca18e731aca48199461f07b2b92aca0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:18.953258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca18e731aca48199461f07b2b92aca0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:19.143001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca18e731aca48199461f07b2b92aca0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:19.152646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca18e731aca48199461f07b2b92aca0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:19.168538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca18e731aca48199461f07b2b92aca0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:19.170975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:19.176312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1009/2042] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:18 2022] DELETE /identity/v3/users/4ca18e731aca48199461f07b2b92aca0 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:19.181688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.186286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.186286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.206236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:19.207744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:19.303308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1034/2043] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:19.315534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.317433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:19.381974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1837640-ba33-4638-b635-cf4695b4d86b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:19.387194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:19.387857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:19.388143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:19.388420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:19.398535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:19.400733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:19.411174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1010/2044] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:19.978345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:19.986675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:20.011596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87788367-33eb-4cf9-b7c2-0d3ee996f719 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.021641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.022318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.022591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.022873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] PATH_INFO: `/v3/projects/8f2962b0b7ef466c81c3db0d6dfc23a9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.025039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2962b0b7ef466c81c3db0d6dfc23a9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.027362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:20.197326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2962b0b7ef466c81c3db0d6dfc23a9', 'request_id': 'req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.202742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:20.204222 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2962b0b7ef466c81c3db0d6dfc23a9', 'request_id': 'req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.383335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2962b0b7ef466c81c3db0d6dfc23a9', 'request_id': 'req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.401522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] The token cache is being invalidate because project 8f2962b0b7ef466c81c3db0d6dfc23a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:20.401813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.402494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.404011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1035/2045] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:19 2022] DELETE /identity/v3/projects/8f2962b0b7ef466c81c3db0d6dfc23a9 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:20.408270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:20.418615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2474d243-12c2-43ec-a3d6-5936c8b01c02 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:20.483549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:20.484273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:20.484470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:20.484738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] PATH_INFO: `/v3/users/185ad21808c34e37a67729e14a76adde` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:20.485876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=185ad21808c34e37a67729e14a76adde)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:20.487851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:20.752862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '185ad21808c34e37a67729e14a76adde'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.771720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '185ad21808c34e37a67729e14a76adde'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.860189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '185ad21808c34e37a67729e14a76adde'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.868757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '185ad21808c34e37a67729e14a76adde'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.882018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5e1fd17f-7f3f-4cf7-81bf-bd927a24c915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '185ad21808c34e37a67729e14a76adde'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:20.889147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1011/2046] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:20 2022] DELETE /identity/v3/users/185ad21808c34e37a67729e14a76adde => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:21.676648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:21.678226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:21.693457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7ca61-36c2-4d88-a55a-e65c9a6140a5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:21.757832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:21.758471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:21.758750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:21.759025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] PATH_INFO: `/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:21.760431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf31e1215b642648a5fbfef4e85c4ce)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:21.762360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:21.928657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf31e1215b642648a5fbfef4e85c4ce', 'request_id': 'req-f6414712-263e-4168-871c-55d0eea981c1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:21.935680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf31e1215b642648a5fbfef4e85c4ce', 'request_id': 'req-f6414712-263e-4168-871c-55d0eea981c1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.008594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf31e1215b642648a5fbfef4e85c4ce', 'request_id': 'req-f6414712-263e-4168-871c-55d0eea981c1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.021097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] The token cache is being invalidate because project 0cf31e1215b642648a5fbfef4e85c4ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:22.021385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.022039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f6414712-263e-4168-871c-55d0eea981c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:22.023585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1036/2047] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:21 2022] DELETE /identity/v3/projects/0cf31e1215b642648a5fbfef4e85c4ce => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:22.029262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.029749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.029853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.049897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:22.051118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.070108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.113832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1012/2048] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:22.122942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.123255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.123536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.147624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:22.148889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.157267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1037/2049] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:22.161719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.171745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de14b5-01e5-41e9-93a8-9eec19f68d52 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.176567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.177294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.177605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.177846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.178915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:22.178915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.181049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.197125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1013/2050] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:22.201600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.211297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b81a008-0606-4693-8367-ce89be4bcae2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.216235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.216847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.217135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.217392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.218036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.219975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.330635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1038/2051] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:22.341248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.352015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ffc182-3ddf-49aa-8d7b-df8054caff54 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.413356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.414113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.414252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.414521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.415833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.418208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.556919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1014/2052] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/users => generated 291 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:22.573031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.587664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55132051-2d91-4d58-b445-397669e6c8f2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.593141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.593891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.594253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.594598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.595383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.597144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.613493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1039/2053] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:22.615266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.626899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e69db0-18e5-4013-adcd-87979d1b3d74 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.632484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.633102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.633380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.633670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.634337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.636015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f829f-101d-48f9-80e5-22e495759d2d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.647197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1015/2054] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:22.651538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.662043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d511b0-dd31-4e8c-821f-1fcb152898a7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.667717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.668580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.668861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.669139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] PATH_INFO: `/v3/projects/b03f396c410c43b3b1422f4611e56a12/users/e27f2701cacc4ed8877eba655d0f89f9/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.685151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03f396c410c43b3b1422f4611e56a12, user_id=e27f2701cacc4ed8877eba655d0f89f9, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.687787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:22.778014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1040/2055] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:22 2022] PUT /identity/v3/projects/b03f396c410c43b3b1422f4611e56a12/users/e27f2701cacc4ed8877eba655d0f89f9/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:22.788653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.791525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.791525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.818923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] MFA Rules not processed for user `e27f2701cacc4ed8877eba655d0f89f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:22.820177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:22.886724 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1016/2056] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:22.891423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:22.901175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:22.955754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097182-2d7b-4a57-8a5f-586ae2939ed9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:22.961121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:22.961732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:22.962061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:22.962333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:22.962985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:22.964975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.156068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1041/2057] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:22 2022] POST /identity/v3/projects => generated 379 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:23.160102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.171057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b447505-6856-4f01-9ec8-4a127f4d39c2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.183863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:23.236793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.237345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.237699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.237870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.238521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.240497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.389669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1017/2058] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/users => generated 296 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:23.393845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.408684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26404735-4910-4097-be16-ab10e002bdc4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.415289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.416563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.417067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.417533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.418688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.421779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.437732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1042/2059] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:23.442501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.453749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcca74-8f3f-4cd3-94a9-ee91af742aa2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.458533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.459262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.459607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.459931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] PATH_INFO: `/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3/users/a8585239b2054a21a53ab3de1c167e25/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.479076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f30043ebbec4ba4b49b33a5e516eaa3, user_id=a8585239b2054a21a53ab3de1c167e25, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.482841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.572568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1018/2060] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:23 2022] PUT /identity/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3/users/a8585239b2054a21a53ab3de1c167e25/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:23.577723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.594904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7aff-7ab3-4032-8e0d-b635dc4114bc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.663758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.664372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.664653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.664973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.669915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.669915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccebb2e-3e77-40a5-a485-0394886ecaaf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.690863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1043/2061] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:23.693761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.704824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8259a9-ccc4-42fa-b153-448c0b39fb5c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.709659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.710326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.710602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.710870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] PATH_INFO: `/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3/users/a8585239b2054a21a53ab3de1c167e25/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.714198 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f30043ebbec4ba4b49b33a5e516eaa3, user_id=a8585239b2054a21a53ab3de1c167e25, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.717041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.730597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1019/2062] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:23 2022] PUT /identity/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3/users/a8585239b2054a21a53ab3de1c167e25/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:23.744732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.745037 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.745313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.789412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] MFA Rules not processed for user `a8585239b2054a21a53ab3de1c167e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:23.791062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.871812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1044/2063] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:23.882572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:23.886722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:23.944523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b3190-1d16-4230-9773-d4a82fb8cefc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:23.948955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224a482-32da-449f-988f-092113669fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:23.949571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a482-32da-449f-988f-092113669fdb service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:23.949851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a482-32da-449f-988f-092113669fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:23.950126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a482-32da-449f-988f-092113669fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:23.959529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224a482-32da-449f-988f-092113669fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:23.960975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224a482-32da-449f-988f-092113669fdb service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:23.970805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1020/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:24.594824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:24.599296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:24.667303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cbe411-4182-411b-97c7-b3eb8f9a7be7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:24.675687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:24.676353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:24.676686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:24.677373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] PATH_INFO: `/v3/users/e27f2701cacc4ed8877eba655d0f89f9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:24.679687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27f2701cacc4ed8877eba655d0f89f9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:24.683765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:24.869951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27f2701cacc4ed8877eba655d0f89f9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.980680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27f2701cacc4ed8877eba655d0f89f9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.981426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27f2701cacc4ed8877eba655d0f89f9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.998636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27f2701cacc4ed8877eba655d0f89f9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:24.999703 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27f2701cacc4ed8877eba655d0f89f9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:25.015929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1045/2065] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:24 2022] DELETE /identity/v3/users/e27f2701cacc4ed8877eba655d0f89f9 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:25.020132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.020851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.020851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.041623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:25.042790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.112576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1021/2066] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:25.123401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:25.124864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.189421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207796-9a05-47c3-9c92-a4a8729b85b9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:25.193577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:25.194263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.194644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.194923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.205148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:25.206602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:25.218436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1046/2067] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:25.801779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:25.803488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:25.818504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a434a-daba-4311-ab4a-204aafc0b63e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:25.825593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:25.826250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:25.826525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:25.826794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] PATH_INFO: `/v3/projects/b03f396c410c43b3b1422f4611e56a12` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:25.828047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03f396c410c43b3b1422f4611e56a12)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:25.830393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:25.844644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:26.013171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03f396c410c43b3b1422f4611e56a12', 'request_id': 'req-9a38753b-bd74-4d3f-996a-629af344c69e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.019420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03f396c410c43b3b1422f4611e56a12', 'request_id': 'req-9a38753b-bd74-4d3f-996a-629af344c69e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.108586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03f396c410c43b3b1422f4611e56a12', 'request_id': 'req-9a38753b-bd74-4d3f-996a-629af344c69e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.121423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] The token cache is being invalidate because project b03f396c410c43b3b1422f4611e56a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:26.121855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.125231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.125231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1022/2068] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:25 2022] DELETE /identity/v3/projects/b03f396c410c43b3b1422f4611e56a12 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:26.128750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:26.139473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e26364-4f80-44c8-bca3-57e4b1341196 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:26.160338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:26.205742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:26.206384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:26.206665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:26.206938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] PATH_INFO: `/v3/users/a8585239b2054a21a53ab3de1c167e25` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:26.208186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8585239b2054a21a53ab3de1c167e25)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:26.210213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:26.450117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8585239b2054a21a53ab3de1c167e25'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.546095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8585239b2054a21a53ab3de1c167e25'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.546719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8585239b2054a21a53ab3de1c167e25'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.561334 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8585239b2054a21a53ab3de1c167e25'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.562078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a898ab8a-527d-41b4-b887-65ca804da12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8585239b2054a21a53ab3de1c167e25'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:26.579019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1047/2069] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:26 2022] DELETE /identity/v3/users/a8585239b2054a21a53ab3de1c167e25 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:27.047741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.049199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:27.072054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38753b-bd74-4d3f-996a-629af344c69e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.151443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:27.152105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.152382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.152656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] PATH_INFO: `/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.153921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f30043ebbec4ba4b49b33a5e516eaa3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:27.155889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:27.330033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f30043ebbec4ba4b49b33a5e516eaa3', 'request_id': 'req-619b4931-8846-4adf-ae80-fc20a0d620e0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:27.337973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f30043ebbec4ba4b49b33a5e516eaa3', 'request_id': 'req-619b4931-8846-4adf-ae80-fc20a0d620e0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:27.523911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f30043ebbec4ba4b49b33a5e516eaa3', 'request_id': 'req-619b4931-8846-4adf-ae80-fc20a0d620e0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:27.538519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] The token cache is being invalidate because project 4f30043ebbec4ba4b49b33a5e516eaa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:27.538795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:27.539493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-619b4931-8846-4adf-ae80-fc20a0d620e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:27.541642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1023/2070] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:27 2022] DELETE /identity/v3/projects/4f30043ebbec4ba4b49b33a5e516eaa3 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:27.551349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.551857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.551947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.574260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:27.575535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:27.669498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1048/2071] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:27.677981 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.678264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.684149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.701938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:27.703186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:27.711946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1024/2072] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:27.716543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.728046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb4c62-6065-4631-8a65-de6ee0588e50 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.733386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:27.734014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.734289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.734680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.735387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:27.736065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:27.737313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:27.751227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1049/2073] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:35:27.755741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.765965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889e050-56a9-4c82-80c8-5ef89ebbc99a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.771505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:27.772125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:27.772451 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:27.772726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:27.773425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:27.776136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:27.935340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:27.943393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1025/2074] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:27.948527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:27.959362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef4f4d-bb3e-4ec7-8bcd-c6aab14ddec7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:27.991879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:28.029534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.030229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.030508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.030784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.031488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.033464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.189574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1050/2075] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:27 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:28.195344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.208116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac7e6c-5f9a-43cd-813e-2b4a7138f633 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.213921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.214623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.214975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.215349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.216285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.218873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.231045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1026/2076] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:28.236737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.257817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556fc6-b224-42b9-a712-8c736c07a659 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.269619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.271039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.271467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.271940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.275391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.276536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d379f-2081-41dd-a54f-eb8f08a45fec admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.309364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.309535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1051/2077] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:28.323519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32cf915-afa1-40ff-90df-592d524e38f5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.330424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.331904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.332608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.332949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] PATH_INFO: `/v3/projects/c654ded9f4084a3c92245815f6198927/users/d52b87189dbb48618c3cedb433e52f32/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.364528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c654ded9f4084a3c92245815f6198927, user_id=d52b87189dbb48618c3cedb433e52f32, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.368623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.504469 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1027/2078] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:28 2022] PUT /identity/v3/projects/c654ded9f4084a3c92245815f6198927/users/d52b87189dbb48618c3cedb433e52f32/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:28.518436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.518436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.518436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.574401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] MFA Rules not processed for user `d52b87189dbb48618c3cedb433e52f32`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:28.575659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.706763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1052/2079] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:28.884442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:28.884442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:28.945177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca1207-c058-4eab-a8ab-bc5e118791c7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:28.949306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:28.949862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:28.950172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:28.950405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:28.959486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:28.960846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2ec19-e1af-4d34-90cc-2e8d9d690e99 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:28.971206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1028/2080] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:28.994288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.078679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a912d-7b46-4a13-b441-c7a4abc9fe84 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.083291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.084058 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.084298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.084559 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] PATH_INFO: `/v3/users/d52b87189dbb48618c3cedb433e52f32` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.085966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52b87189dbb48618c3cedb433e52f32)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.089105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.370712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:29.378636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52b87189dbb48618c3cedb433e52f32'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.512854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52b87189dbb48618c3cedb433e52f32'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.513555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52b87189dbb48618c3cedb433e52f32'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.529337 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52b87189dbb48618c3cedb433e52f32'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.530078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52b87189dbb48618c3cedb433e52f32'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:29.543122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1053/2081] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:28 2022] DELETE /identity/v3/users/d52b87189dbb48618c3cedb433e52f32 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:29.547780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.548300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.548416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.569814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:29.571126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.645111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1029/2082] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:29.656519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:29.660788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:29.723175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854776-7106-4c73-99ab-06fe855a1db4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:29.727238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:29.728209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:29.728699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:29.728962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:29.739731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:29.741268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0839e-9cf0-4b57-8d9e-45dab13ccf35 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:29.752884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1054/2083] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:30.359627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.361121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:30.373889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d38ce-e7a7-41aa-ba33-a0a63408ef69 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.378715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.379393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.379673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.379947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] PATH_INFO: `/v3/projects/c654ded9f4084a3c92245815f6198927` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.381090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c654ded9f4084a3c92245815f6198927)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.383044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.547687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c654ded9f4084a3c92245815f6198927', 'request_id': 'req-9b022434-0930-454d-9ba0-decc23a7436f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:30.554175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c654ded9f4084a3c92245815f6198927', 'request_id': 'req-9b022434-0930-454d-9ba0-decc23a7436f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:30.666100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c654ded9f4084a3c92245815f6198927', 'request_id': 'req-9b022434-0930-454d-9ba0-decc23a7436f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:30.667897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:30.679061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] The token cache is being invalidate because project c654ded9f4084a3c92245815f6198927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:30.679383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:30.680073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b022434-0930-454d-9ba0-decc23a7436f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:30.682503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1030/2084] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:30 2022] DELETE /identity/v3/projects/c654ded9f4084a3c92245815f6198927 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:30.687847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.688424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.688424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.710200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:30.711464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:30.775282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1055/2085] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:30.782801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.783084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.783379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.804552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:30.806422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:30.814682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1031/2086] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:30.819335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.829981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e94ffc-7cec-4ca0-8e23-bc64bf790f4e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.834995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.835629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.835910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.836213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.837502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:30.837815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.839436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:30.854760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1056/2087] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:30.858048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:30.867484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75732fcf-fd4a-4951-ae15-125fb4448715 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:30.873082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:30.879296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:30.879296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:30.879296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:30.879296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:30.879296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.025531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1032/2088] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:30 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:31.030472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.042527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5537c6-2b92-4c16-bc04-91e528515802 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.099298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:31.114433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.115640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.115640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.124141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.124141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.124141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.349217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1057/2089] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:31 2022] POST /identity/v3/users => generated 291 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:31.350610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.361787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd922ffa-7d47-41f5-8e19-fc6e825923ae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.366385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.366991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.367295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.367568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.368717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.370389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.381566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1033/2090] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:31.386919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.398351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eef92f-c0ef-42e4-b87b-e9fbe296720d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.406209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.406827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.407275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.407781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.408384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.410133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59051ab-c1c4-4870-81d7-581eb1d6a297 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.422383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1058/2091] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:31.426760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.437823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6fdeb-38dd-420b-8255-a32975b8fff7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.442698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.443401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.443671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.443938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] PATH_INFO: `/v3/projects/98240e01aaa84f3e8c7e425ad2c9eb4f/users/3a32dcea37fb4255823fec5eb5c57e00/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.459573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98240e01aaa84f3e8c7e425ad2c9eb4f, user_id=3a32dcea37fb4255823fec5eb5c57e00, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.462445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.556428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1034/2092] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:31 2022] PUT /identity/v3/projects/98240e01aaa84f3e8c7e425ad2c9eb4f/users/3a32dcea37fb4255823fec5eb5c57e00/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:31.559348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.559634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.559930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.589756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:31.591092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:31.661329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1059/2093] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:31.672148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.673466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:31.733627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd8f761-4cec-4fc3-ac5c-302da89af177 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.737546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.738185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.738446 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.738717 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.740503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:31.741904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:31.749014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.750473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9619f5-83a9-41f5-9643-860f192ca4ff service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:31.754490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d8c39-64aa-49a6-a759-b111a9d8a3f9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:31.761552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1035/2094] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:31.764405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:31.765054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:31.765333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:31.773719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] PATH_INFO: `/v3/users/4ae85b5e67574e1c9ffaf6b6445ff414` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:31.773719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae85b5e67574e1c9ffaf6b6445ff414)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:31.773719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:32.031390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae85b5e67574e1c9ffaf6b6445ff414'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:32.189682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae85b5e67574e1c9ffaf6b6445ff414'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:32.190338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae85b5e67574e1c9ffaf6b6445ff414'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:32.209356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae85b5e67574e1c9ffaf6b6445ff414'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:32.210149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae85b5e67574e1c9ffaf6b6445ff414'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:32.224008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1060/2095] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:31 2022] DELETE /identity/v3/users/4ae85b5e67574e1c9ffaf6b6445ff414 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:32.229123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:32.229418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:32.229694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:32.250404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:32.251850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:32.316549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1036/2096] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:32.326973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:32.328583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:32.376832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:32.396304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124e06-aff9-455d-96a0-99129e1f5f52 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:32.400539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:32.401153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:32.401431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:32.401839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:32.413709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:32.415993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30055e-b28c-47b4-9071-c1e744151e0b service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:32.427852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1061/2097] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:33.066064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.070859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.080324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf02954-2bcd-4ca8-bc88-f4bc9aad8e0c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.085002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.085640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.085997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.086198 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] PATH_INFO: `/v3/projects/66808b6f25e74338a17b1ac03568290c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.087538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66808b6f25e74338a17b1ac03568290c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.089481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.117200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:33.129005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.129489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.129795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.154088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:33.159840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.216111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1062/2098] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:33.238770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66808b6f25e74338a17b1ac03568290c', 'request_id': 'req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:33.246846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66808b6f25e74338a17b1ac03568290c', 'request_id': 'req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:33.266764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.268239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.335005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66808b6f25e74338a17b1ac03568290c', 'request_id': 'req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:33.341828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a827-cdf4-40f3-b606-8d8c432d9a7a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.347123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] The token cache is being invalidate because project 66808b6f25e74338a17b1ac03568290c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:33.347447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:33.348157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcce9b2-a220-45ff-9dc1-bcdb9e61d8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:33.349752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1037/2099] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:33 2022] DELETE /identity/v3/projects/66808b6f25e74338a17b1ac03568290c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:33.355440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee52d38-b1d2-4c04-92b6-f9440db347a7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.355723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee52d38-b1d2-4c04-92b6-f9440db347a7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.356063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee52d38-b1d2-4c04-92b6-f9440db347a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.375632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aee52d38-b1d2-4c04-92b6-f9440db347a7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:33.377103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aee52d38-b1d2-4c04-92b6-f9440db347a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.410429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.411181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.411387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.411643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.413587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.465304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1038/2100] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:33.473907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.474282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.474746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.483807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.485725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.498095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1063/2101] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:35:33.499102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:33.501899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.512318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1039/2102] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:33.519060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.534452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c92a5-da58-4a10-8fb7-5d282f21e52a service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.542969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.543754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.547518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.547815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.548514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:33.548824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.550482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7411c59-4d39-4c60-b477-f4f8446f2b2c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.568788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1064/2103] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:33.573873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.585596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb2029c-dc59-41f0-8044-a292806b8d86 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.590933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.591665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.592035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.592380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.593250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.595393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.614272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.614790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.615257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.645775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] MFA Rules not processed for user `3a32dcea37fb4255823fec5eb5c57e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:33.647303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.685942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:33.716924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1065/2104] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:33.738293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.739748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:33.759651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1040/2105] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:33.760992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:33.810309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948e2fb-ab7e-46a0-8db6-66635940e8f2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.827418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411edac7-0a9c-4284-9ea4-6f68f7160ed0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:33.832152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.832762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.833037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.833307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.833989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.835973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.861411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:33.862217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:33.862384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:33.862652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:33.918955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:33.920328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:33.929196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1066/2106] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:33.984632 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1041/2107] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:33 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:33.989905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.017949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fe00c-e904-47f3-8445-0c847f143a6f service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.034939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.035636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.035954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.036226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.037127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.038951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.052903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1067/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:34.055664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.066366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5996d-618d-42d0-8d74-9cf1fdc29ebc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.071316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.072284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.072629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.073013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.073728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.075444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4dd02-41cd-4692-93c8-5cdb819aedd1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.085507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1042/2109] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:34.091177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.101445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f48-7a5a-483d-bf39-8b11c5ea47f9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.106990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.107668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.107971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.108245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] PATH_INFO: `/v3/projects/fa4f14d16eeb439599f928e4267bb3d6/users/6064130f907f42579f8bb02338f7fa92/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.123620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4f14d16eeb439599f928e4267bb3d6, user_id=6064130f907f42579f8bb02338f7fa92, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.126465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.283717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1068/2110] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:34 2022] PUT /identity/v3/projects/fa4f14d16eeb439599f928e4267bb3d6/users/6064130f907f42579f8bb02338f7fa92/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:34.298235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.298430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.298709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.322998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:34.338007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] MFA Rules not processed for user `6064130f907f42579f8bb02338f7fa92`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:34.339438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:34.419758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1043/2111] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:34.424493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.501943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2db1a5-a225-4aac-8b62-13f7f18b5d52 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.506746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.507851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.508059 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.508328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.509583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.511298 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d7716-20dc-4a6f-bb45-5e79b8ed5712 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.632372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:34.642949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1069/2112] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:34 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:35:34.644189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.655824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526922f-bdae-4ae8-9642-cc6b57f9355f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.700332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.700332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.700332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.722923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:34.725167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:34.727233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.742998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.742998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.742998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.742998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.742998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.823793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1070/2113] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:35:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:34.836784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.838314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:34.852610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:34.870269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1044/2114] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:34 2022] POST /identity/v3/users => generated 292 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:34.874750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.885466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce00f05-a708-4d53-9a9d-e0d738d45fa9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.891334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.892325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.892752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.893187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.893894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.896044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.911670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1045/2115] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:34.912836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.924861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b3b4b-4683-4086-b5c3-3d01c159624c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.932623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.933455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.933743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.934319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.934889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.936829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.937616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29fa92-e5e1-4c1b-b522-cbe463adf898 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.949068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.950159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.950844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.951311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.951606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1046/2116] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:34.954701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:34.967287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.969215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b449fb26-69b6-4bea-a9eb-945f20b19eb8 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:34.969613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ac3cad-228e-46d1-ae7c-86a892fb3955 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:34.974891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:34.981233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:34.981233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:34.981233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] PATH_INFO: `/v3/projects/07e2033c15f745fea31cda3dc45b2f95/users/f7119deaa19448eeb80f78222d9b364b/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:34.981722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1071/2117] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:35:34.994089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e2033c15f745fea31cda3dc45b2f95, user_id=f7119deaa19448eeb80f78222d9b364b, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:34.998868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.230522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1047/2118] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:34 2022] PUT /identity/v3/projects/07e2033c15f745fea31cda3dc45b2f95/users/f7119deaa19448eeb80f78222d9b364b/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:35.242283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.242591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.242861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.273906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] MFA Rules not processed for user `f7119deaa19448eeb80f78222d9b364b`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:35.275453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:35.342831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1072/2119] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:35.347648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.413990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d499-742c-4e65-a3d8-0eb0907bd48f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.419812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.420433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.420712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.420982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.421650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.423660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.599718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1048/2120] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:35 2022] POST /identity/v3/projects => generated 383 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:35.604775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.616615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605812c3-30d6-4033-80fd-c61f8ea747b1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.678223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.678905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.679123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.679432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.680089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.682871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.789765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1073/2121] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:35 2022] POST /identity/v3/users => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:35.794952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.810104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a321-61dd-4728-b51d-d93d09499d43 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.817904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.818530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.818840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.819105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.819795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.822381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.835886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1049/2122] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:35:35.842882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:35.855405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32408e21-f3cb-46df-a430-ab1dbf6ae281 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:35.861363 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:35.862123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:35.862438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:35.862684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] PATH_INFO: `/v3/projects/c45502573669485988be4be631accdf2/users/ddc94fe874414890a4710ef8b8a37721/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:35.880997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45502573669485988be4be631accdf2, user_id=ddc94fe874414890a4710ef8b8a37721, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:35.884815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:35.993355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:36.003180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1074/2123] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:35 2022] PUT /identity/v3/projects/c45502573669485988be4be631accdf2/users/ddc94fe874414890a4710ef8b8a37721/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:36.009995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.026439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:36.031241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440405c-0845-42fc-bebb-473f42e8ee07 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.126562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.127106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.127415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.127682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.128318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.130230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76ec7a-56b5-4993-8aaf-ebe4e04f7f74 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.142657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1050/2124] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:36.147695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.163828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147ebd-2e1e-4e85-b2ba-bda5ef08c78e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.170125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.172557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.172557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.172557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] PATH_INFO: `/v3/projects/c45502573669485988be4be631accdf2/users/ddc94fe874414890a4710ef8b8a37721/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.174716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c45502573669485988be4be631accdf2, user_id=ddc94fe874414890a4710ef8b8a37721, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.178027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514cae3-2131-48a3-9d59-0b0370fa5612 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.181465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.181972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.182405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.195235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1075/2125] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:36 2022] PUT /identity/v3/projects/c45502573669485988be4be631accdf2/users/ddc94fe874414890a4710ef8b8a37721/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:36.209348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.209628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.209902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.210883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:36.213054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.248264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] MFA Rules not processed for user `ddc94fe874414890a4710ef8b8a37721`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:36.249984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.287351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1051/2126] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:36.301664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.345009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1076/2127] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:36.361988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:36.363475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:36.382732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362b90d-cd9e-486f-8125-4fde069ea938 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.393017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.398139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.398139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.398139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.406151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.408338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759ae49-5321-4495-9581-d6a3c9f97297 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.420181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:36.423862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1052/2128] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:35:36.455367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091d38c-7857-44e5-bad7-ce670d5da511 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:36.461722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:36.463939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:36.464387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:36.464840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:36.535915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:36.537460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:36.548410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1077/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:35:37.170746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c1c-3113-4d54-9471-2fe98b0617fb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.170746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c1c-3113-4d54-9471-2fe98b0617fb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.170746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef8c1c-3113-4d54-9471-2fe98b0617fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.196309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8eef8c1c-3113-4d54-9471-2fe98b0617fb None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:37.198273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8eef8c1c-3113-4d54-9471-2fe98b0617fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:37.206216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1053/2130] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:35:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:37.219734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:37.237828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6357691a-f79e-4b9a-a106-fb4ab6cd1aaa service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:37.242990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:37.244720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:37.244995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:37.245258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:37.254001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:37.255698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:37.265776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1078/2131] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:35:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:35:39.334324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:39.335128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:39.335128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:39.366471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] MFA Rules not processed for user `3a32dcea37fb4255823fec5eb5c57e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:39.366471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:39.456723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1054/2132] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:40.030328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.040089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000eaed-6fae-49b6-b3ee-a1fde94acb6a service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.043940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.044721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.045067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.045410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.054666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:40.107526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.108961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf409a4-70be-49a7-b3a8-ba124d364f75 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:40.119710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1079/2133] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:40.901966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:40.903681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:40.979679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed67a4-f497-44b4-8438-91591e82a1dd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:40.987363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:40.988044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:40.988326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:40.988600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] PATH_INFO: `/v3/users/a8a6a1293fe44530a07d8bc4b0f74613` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:40.989790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a6a1293fe44530a07d8bc4b0f74613)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:40.991786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.415524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a6a1293fe44530a07d8bc4b0f74613'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:41.416296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a6a1293fe44530a07d8bc4b0f74613'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:41.524157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a6a1293fe44530a07d8bc4b0f74613'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:41.534236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a6a1293fe44530a07d8bc4b0f74613'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:41.550213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a6a1293fe44530a07d8bc4b0f74613'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:41.551904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:41.560233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1055/2134] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:40 2022] DELETE /identity/v3/users/a8a6a1293fe44530a07d8bc4b0f74613 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:41.570474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.571393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.571393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.649121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:41.651504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:41.752941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1080/2135] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:35:41.766846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:41.768541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:41.851532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bad86-d577-4846-b421-915c93e95572 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:41.853300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:41.853965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:41.854348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:41.854937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:41.863914 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:41.865349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:41.875495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1056/2136] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:42.234533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:42.236117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:42.249629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5885b-afc9-4999-a46a-29cd963cde90 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:42.254283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:42.254919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:42.255215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:42.255598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] PATH_INFO: `/v3/projects/22dcb2335f19495e91b8de154d2110ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:42.257507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dcb2335f19495e91b8de154d2110ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:42.259215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:42.477808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dcb2335f19495e91b8de154d2110ac', 'request_id': 'req-c369c92d-5f73-47db-bf72-954b8ae1b845'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:42.484524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dcb2335f19495e91b8de154d2110ac', 'request_id': 'req-c369c92d-5f73-47db-bf72-954b8ae1b845'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:42.569870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dcb2335f19495e91b8de154d2110ac', 'request_id': 'req-c369c92d-5f73-47db-bf72-954b8ae1b845'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:42.582680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] The token cache is being invalidate because project 22dcb2335f19495e91b8de154d2110ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:42.583392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:42.583650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:42.585214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1081/2137] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:42 2022] DELETE /identity/v3/projects/22dcb2335f19495e91b8de154d2110ac => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:42.908559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:42.911030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.000611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df15b2-2471-43c0-9926-b6ad6ab57602 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:43.065821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:43.066861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:43.067418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:43.068172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:43.071273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:43.149248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:43.150782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01679f-9ccc-4882-8c5a-518641f04446 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:43.161569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1057/2138] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:43.997009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:43.998808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.038295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:44.090602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369c92d-5f73-47db-bf72-954b8ae1b845 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.095426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.096069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.096342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.096609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] PATH_INFO: `/v3/users/b08f847ded484114bd3897cf07a48f1d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.097731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08f847ded484114bd3897cf07a48f1d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.099753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.525265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08f847ded484114bd3897cf07a48f1d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.606478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08f847ded484114bd3897cf07a48f1d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.607247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08f847ded484114bd3897cf07a48f1d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.621387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08f847ded484114bd3897cf07a48f1d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.621855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08f847ded484114bd3897cf07a48f1d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:44.635173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1082/2139] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:43 2022] DELETE /identity/v3/users/b08f847ded484114bd3897cf07a48f1d => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:44.640618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.641332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.641332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.655363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:44.662841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:44.664118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.733459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1058/2140] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:44.743350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:44.745053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:44.809328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3fc27b-116b-40ca-8ade-cfd546221fa4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:44.813003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:44.813615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:44.813895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:44.814167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:44.823930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:44.825427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07feb63-6354-4a0e-8d65-9f5e1fab2965 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:44.836086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1083/2141] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:45.397338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.398839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.420750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e93ad1-b2f7-42fa-a751-1598ae72598f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.445462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.445628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.445919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.446787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] PATH_INFO: `/v3/projects/c666a18b5b454c16b0a357ecf11f0146` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.448022 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c666a18b5b454c16b0a357ecf11f0146)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.450826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.662680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c666a18b5b454c16b0a357ecf11f0146', 'request_id': 'req-601c01e4-dea4-40bf-98d5-459599ed0079'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.670850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c666a18b5b454c16b0a357ecf11f0146', 'request_id': 'req-601c01e4-dea4-40bf-98d5-459599ed0079'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.761218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c666a18b5b454c16b0a357ecf11f0146', 'request_id': 'req-601c01e4-dea4-40bf-98d5-459599ed0079'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.773263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] The token cache is being invalidate because project c666a18b5b454c16b0a357ecf11f0146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:45.773552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.774195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-601c01e4-dea4-40bf-98d5-459599ed0079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:45.775734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1059/2142] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:45 2022] DELETE /identity/v3/projects/c666a18b5b454c16b0a357ecf11f0146 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:45.781022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.781523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.781617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.803953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:45.805201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.861941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:45.881872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1084/2143] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:45.884067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.884432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.884759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.906502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:45.907688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:45.915637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1060/2144] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:45.920507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.930162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6c8d3-4988-4912-9de5-35b47bedb0ec None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.935510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.936184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.936523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.936734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.937408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:45.937709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.939386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:45.952285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1085/2145] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:45.956748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:45.967557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11534d7-2f7c-49fd-bdd1-8005c47168e4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:45.972943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:45.985966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:45.985966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:45.985966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:45.985966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:45.985966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.146597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1061/2146] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:45 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:46.157427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.169066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d54ff-2c77-4cea-aa13-0dbb520ea1c5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.233084 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.233711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.233989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.234260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.234904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.236866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.405174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1086/2147] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:46 2022] POST /identity/v3/users => generated 283 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:46.409492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.419925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2027e-c4b6-4bf6-976a-63ac89b14308 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.424832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.425426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.425706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.425976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.426627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.428308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.441881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1062/2148] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:46.545969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.559894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa60b08-8f30-4bde-bbed-dfff83f386a4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.564087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.564834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.565108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.565413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.566180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.568555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7946b-0fc9-4e31-ab6a-dc0dd565d4c4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:46.586354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:46.589475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1087/2149] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:46.598535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feb9d8-8cd3-4e82-a9dc-bf250e2c8964 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:46.603641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:46.603938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:46.604220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:46.604489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] PATH_INFO: `/v3/projects/1ea4a9f1a3fa4892bbe2c2b71f7de201/users/03bc7b9c345a465fa7cd5d053ece9751/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:46.619644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea4a9f1a3fa4892bbe2c2b71f7de201, user_id=03bc7b9c345a465fa7cd5d053ece9751, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:46.622429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:47.239405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1063/2150] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:46 2022] PUT /identity/v3/projects/1ea4a9f1a3fa4892bbe2c2b71f7de201/users/03bc7b9c345a465fa7cd5d053ece9751/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:47.250291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:47.250796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:47.250889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:47.288179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] MFA Rules not processed for user `03bc7b9c345a465fa7cd5d053ece9751`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:47.289538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:47.370386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1088/2151] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:47.394141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:47.396649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:47.434857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:47.464519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc85de-4add-4f61-a3b0-00b8a23d0dd0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:47.469530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:47.470112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:47.470392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:47.470665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:47.479640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:47.481812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1b097-3c3f-4515-9dd2-8d44c40c0668 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:47.491479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1064/2152] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:49.400820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:49.403528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:49.462958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:49.478104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252f931-a786-4c5b-b157-5019664dfa02 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:49.483379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:49.483786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.484166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.484323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] PATH_INFO: `/v3/users/6064130f907f42579f8bb02338f7fa92` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.485595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6064130f907f42579f8bb02338f7fa92)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:49.487408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:49.797651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6064130f907f42579f8bb02338f7fa92'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:49.940629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6064130f907f42579f8bb02338f7fa92'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:49.941310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6064130f907f42579f8bb02338f7fa92'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:49.955861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6064130f907f42579f8bb02338f7fa92'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:49.956605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6064130f907f42579f8bb02338f7fa92'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:49.969658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1089/2153] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:49 2022] DELETE /identity/v3/users/6064130f907f42579f8bb02338f7fa92 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:49.975404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607dd3-ebb7-468d-bea6-0e151835c116 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:49.975933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607dd3-ebb7-468d-bea6-0e151835c116 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:49.976048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c607dd3-ebb7-468d-bea6-0e151835c116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:49.995084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1c607dd3-ebb7-468d-bea6-0e151835c116 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:49.997153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c607dd3-ebb7-468d-bea6-0e151835c116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.056606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1065/2154] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:50.067260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.069147 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.134455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca873fb-0b21-44c1-89d9-c2c0a18a328a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.138680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.139450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.139793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.140139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.150620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.152112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.163983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1090/2155] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:35:50.423423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.423786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.424118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.446656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] MFA Rules not processed for user `8937e9f80f1540e389ee283971624741`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:50.447946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.515715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1066/2156] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:50.721523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:50.724848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:50.755889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05c6e3-15ae-4a21-96f4-b80c8713eb20 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:50.763664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:50.764673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:50.764958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:50.765235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] PATH_INFO: `/v3/projects/fa4f14d16eeb439599f928e4267bb3d6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:50.766675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4f14d16eeb439599f928e4267bb3d6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:50.768617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:50.900761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4f14d16eeb439599f928e4267bb3d6', 'request_id': 'req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:50.908305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4f14d16eeb439599f928e4267bb3d6', 'request_id': 'req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.003187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4f14d16eeb439599f928e4267bb3d6', 'request_id': 'req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.023348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:51.033981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] The token cache is being invalidate because project fa4f14d16eeb439599f928e4267bb3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:51.034856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.035862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.046713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1091/2157] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:50 2022] DELETE /identity/v3/projects/fa4f14d16eeb439599f928e4267bb3d6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:51.051709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:51.065807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd831b-0a04-43aa-af43-1fca482c89cf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:51.144438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:51.145079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:51.145351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:51.145804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] PATH_INFO: `/v3/users/f7119deaa19448eeb80f78222d9b364b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:51.147413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7119deaa19448eeb80f78222d9b364b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:51.149334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:51.174094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:51.680967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7119deaa19448eeb80f78222d9b364b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.681626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7119deaa19448eeb80f78222d9b364b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.835663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7119deaa19448eeb80f78222d9b364b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.845569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7119deaa19448eeb80f78222d9b364b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.859490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7119deaa19448eeb80f78222d9b364b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:51.866070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1067/2158] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:51 2022] DELETE /identity/v3/users/f7119deaa19448eeb80f78222d9b364b => generated 0 bytes in 815 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:52.497153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.498745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.514936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b7e8d-a71f-4ae8-93fe-08fe36ffc417 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.589622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.590955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.591269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.591536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] PATH_INFO: `/v3/projects/07e2033c15f745fea31cda3dc45b2f95` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.592695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e2033c15f745fea31cda3dc45b2f95)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.594719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:52.738576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e2033c15f745fea31cda3dc45b2f95', 'request_id': 'req-4d0500b5-33da-46eb-bcac-b987e06a8410'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.745726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e2033c15f745fea31cda3dc45b2f95', 'request_id': 'req-4d0500b5-33da-46eb-bcac-b987e06a8410'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.878685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e2033c15f745fea31cda3dc45b2f95', 'request_id': 'req-4d0500b5-33da-46eb-bcac-b987e06a8410'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.901466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] The token cache is being invalidate because project 07e2033c15f745fea31cda3dc45b2f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:52.901466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.902467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:52.911010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1092/2159] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:52 2022] DELETE /identity/v3/projects/07e2033c15f745fea31cda3dc45b2f95 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:52.911517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:52.913161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:52.929882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf19714-2a20-4c03-b285-23be1599d432 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:52.988875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:52.989420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:52.989694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:52.989963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] PATH_INFO: `/v3/users/ddc94fe874414890a4710ef8b8a37721` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:52.991103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc94fe874414890a4710ef8b8a37721)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:52.993043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:53.333327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc94fe874414890a4710ef8b8a37721'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:53.333985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc94fe874414890a4710ef8b8a37721'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:53.409522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc94fe874414890a4710ef8b8a37721'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:53.417860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc94fe874414890a4710ef8b8a37721'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:53.434118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b9c8df9-e87f-4335-9a3d-5bdb13da179c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc94fe874414890a4710ef8b8a37721'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:53.441202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1068/2160] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:35:52 2022] DELETE /identity/v3/users/ddc94fe874414890a4710ef8b8a37721 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:54.051397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.052894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.068794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0500b5-33da-46eb-bcac-b987e06a8410 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.133303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.133948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.134362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.134637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] PATH_INFO: `/v3/projects/c45502573669485988be4be631accdf2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.135790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c45502573669485988be4be631accdf2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.138166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.300831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.316915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c45502573669485988be4be631accdf2', 'request_id': 'req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.324229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c45502573669485988be4be631accdf2', 'request_id': 'req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.438138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c45502573669485988be4be631accdf2', 'request_id': 'req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.451095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] The token cache is being invalidate because project c45502573669485988be4be631accdf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:35:54.451450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.452446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7e9d6b51-54c2-4582-bb2b-0c01f01d0b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:35:54.453901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1093/2161] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:35:54 2022] DELETE /identity/v3/projects/c45502573669485988be4be631accdf2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:35:54.459259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.459774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.459774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.481555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:54.483182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.492623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:35:54.551143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1069/2162] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:54.560447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.560782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.561096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.582248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:54.583534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:54.592818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1094/2163] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:54.597374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.608233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3d0ef-51f9-41dc-9564-2b99234b9198 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.613106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.613710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.614034 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.614282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.614924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:35:54.615248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.616855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.629142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1070/2164] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:35:54.633662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.644022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177fca4-0f38-4a1b-8102-a3df61d746c6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.649589 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.650198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.650479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.650749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.651440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.654149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:54.809214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1095/2165] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:54.810566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:54.822088 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fb911-bbbf-4894-9eab-d1d1b418e806 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:54.922858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:54.924502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:54.933438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:54.933438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:54.933910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:54.936166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.044770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1071/2166] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:35:54 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:35:55.048424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.075247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4546c4-4a1c-4f0d-9c84-09a669de4cd5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.087246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.088191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.088627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.089045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.090391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.092634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.117565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1096/2167] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:55.118591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.140924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d17f4-f39d-478c-adc6-e9f865fd5b72 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.148527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.149372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.149842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.150802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.151466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.154937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2303e94-67a5-463e-95d1-c373dc145ae5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.175426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1072/2168] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:35:55.180013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.193353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224977ac-6d26-4b13-bdbf-b964287f6648 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.198371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.204727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.204727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.204727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] PATH_INFO: `/v3/projects/9193c40d930d4774b800ddfeefbf8c92/users/cf9862b97ba64cc09b4903ba24ac2181/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.218231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9193c40d930d4774b800ddfeefbf8c92, user_id=cf9862b97ba64cc09b4903ba24ac2181, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.221024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.365214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1097/2169] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:35:55 2022] PUT /identity/v3/projects/9193c40d930d4774b800ddfeefbf8c92/users/cf9862b97ba64cc09b4903ba24ac2181/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:35:55.370607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd79c2-1729-4dda-ba5b-c9a01e39a240 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.370890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd79c2-1729-4dda-ba5b-c9a01e39a240 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.371204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd79c2-1729-4dda-ba5b-c9a01e39a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.396363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8dcd79c2-1729-4dda-ba5b-c9a01e39a240 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:35:55.397605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8dcd79c2-1729-4dda-ba5b-c9a01e39a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.462873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1073/2170] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:35:55.474091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:35:55.487299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:35:55.548079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b662561-f612-4be6-87be-de8c435973ce admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:35:55.552236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:35:55.555708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:35:55.555708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:35:55.555708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:35:55.563799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:35:55.565238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:35:55.576028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1098/2171] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:03.011838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.012586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.012729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.046367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] MFA Rules not processed for user `cf9862b97ba64cc09b4903ba24ac2181`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:03.047622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:03.057312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.131192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1074/2172] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:03.133538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.145043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eadbd3-1949-439f-8cea-829c067c3731 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.150205 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.150815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.151097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.151439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.162546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.162546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.287831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1099/2173] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:03.291816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.303401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1f2f2-0451-4d0d-b2cb-fd2afc17923e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.368108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.368714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.368988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.369259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.369888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.372762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.493134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1075/2174] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:36:03 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:36:03.503258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.516482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760bc81-1874-47f8-afe2-b42ad5a3e10e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.521190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.521831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.522106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.522375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.523024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.524669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.541130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1100/2175] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:03.546277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.556730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efe876-d9cc-42b8-8d49-d02116536d7c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.562090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.562742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.563090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.563398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] PATH_INFO: `/v3/projects/df1b6a3f534f4ec682f5f73c1292124f/users/086c17f07797436b85b8dd6615112aaf/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.580628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1b6a3f534f4ec682f5f73c1292124f, user_id=086c17f07797436b85b8dd6615112aaf, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.583401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.705328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1076/2176] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:03 2022] PUT /identity/v3/projects/df1b6a3f534f4ec682f5f73c1292124f/users/086c17f07797436b85b8dd6615112aaf/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:03.709004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.720464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25e3dd-502c-4f11-b1ca-ba56ff9e1c34 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.739599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:03.780017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.780629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.780907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.781175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.781818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.783482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35dcd20-8ebf-4de3-bebf-0a800e36b8eb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.794201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1101/2177] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:36:03.799103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:03.813162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fde890-90f3-422d-bf90-e1d633dfee14 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:03.816378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:03.817042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:03.817355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:03.817623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] PATH_INFO: `/v3/projects/df1b6a3f534f4ec682f5f73c1292124f/users/086c17f07797436b85b8dd6615112aaf/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:03.820043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df1b6a3f534f4ec682f5f73c1292124f, user_id=086c17f07797436b85b8dd6615112aaf, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:03.822736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:03.835607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1077/2178] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:36:03 2022] PUT /identity/v3/projects/df1b6a3f534f4ec682f5f73c1292124f/users/086c17f07797436b85b8dd6615112aaf/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:36:11.736595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a296fe-4c92-4b87-a70e-cc0ab004ffdd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.737154 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a296fe-4c92-4b87-a70e-cc0ab004ffdd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.737694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a296fe-4c92-4b87-a70e-cc0ab004ffdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.771892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a8a296fe-4c92-4b87-a70e-cc0ab004ffdd None None] MFA Rules not processed for user `086c17f07797436b85b8dd6615112aaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:11.773317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8a296fe-4c92-4b87-a70e-cc0ab004ffdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:11.880669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1102/2179] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:11.895809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:11.897709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:11.970783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a978a5-5a74-4a33-925a-d4111a594388 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:11.974603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:11.975276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:11.975542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:11.975907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:11.997811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:12.052901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:12.054579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:12.065424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1078/2180] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:12.406279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b95aeb-ca7f-4eb3-bd37-08296ca6adca None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:12.406563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b95aeb-ca7f-4eb3-bd37-08296ca6adca None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:12.406838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b95aeb-ca7f-4eb3-bd37-08296ca6adca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:12.427630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-60b95aeb-ca7f-4eb3-bd37-08296ca6adca None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:12.428792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60b95aeb-ca7f-4eb3-bd37-08296ca6adca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:12.501814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1103/2181] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:12.514456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:12.516878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:12.608869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d6beb-a1fa-4959-8537-e27eac5511fe service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:12.615216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:12.615860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:12.616140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:12.616415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:12.625807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:12.627495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:12.635932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1079/2182] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:22.493999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=78 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:22.632584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=52 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:25.532989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb720a3-adbe-4581-af3b-445b58e27433 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:25.533676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb720a3-adbe-4581-af3b-445b58e27433 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:25.533676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb720a3-adbe-4581-af3b-445b58e27433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:25.554116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5eb720a3-adbe-4581-af3b-445b58e27433 None None] MFA Rules not processed for user `cf9862b97ba64cc09b4903ba24ac2181`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:25.555505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5eb720a3-adbe-4581-af3b-445b58e27433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:25.563207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1104/2183] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:36:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:36:28.168826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:28.170271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:28.249296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aebd49-2bfc-4e79-8954-343d82cbd3a1 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:28.254882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:28.255526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:28.255801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:28.256070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:28.258108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:28.271534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:28.275249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:28.286483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1080/2184] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:36:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:36:30.520945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.521607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.521607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.542690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:36:30.543952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:36:30.551617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1105/2185] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:36:30.565228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:36:30.576709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d820d19-f53f-42a4-b356-d651b7f52ef8 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:36:30.583756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:36:30.584803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:36:30.585308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:36:30.585780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:36:30.594818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:36:30.597257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59351318-c530-47c5-9d89-28296dc3199e service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:36:30.605323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1081/2186] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:36:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:36:40.536774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:36:40.602163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=38 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:00.573533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:00.573533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:00.699887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dc15d-5db5-4cb9-a29f-f77a0c1ea2fd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:00.712461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:00.713129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:00.713413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:00.713796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] PATH_INFO: `/v3/users/03bc7b9c345a465fa7cd5d053ece9751` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:00.717512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03bc7b9c345a465fa7cd5d053ece9751)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:00.722659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:00.997436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03bc7b9c345a465fa7cd5d053ece9751'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.097106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03bc7b9c345a465fa7cd5d053ece9751'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.097724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03bc7b9c345a465fa7cd5d053ece9751'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.110385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03bc7b9c345a465fa7cd5d053ece9751'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.111538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03bc7b9c345a465fa7cd5d053ece9751'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:01.124182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1106/2187] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:00 2022] DELETE /identity/v3/users/03bc7b9c345a465fa7cd5d053ece9751 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:01.129569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.130101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.130101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.150404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:01.151642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.218947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1082/2188] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:01.231508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.233022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:01.304640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cb96de-f017-4e9a-be4e-b9bcb622121e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:01.308792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:01.309574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:01.310004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:01.310445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:01.321582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:01.323314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596279-5c06-4ccc-aa88-804345f36cd0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:01.335335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1107/2189] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:01.987958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:01.991409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.002560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e95bfe6-e6ad-42d3-b9f3-78aec856c3a1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.007569 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.008154 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.008463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.008869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] PATH_INFO: `/v3/projects/1ea4a9f1a3fa4892bbe2c2b71f7de201` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.010056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea4a9f1a3fa4892bbe2c2b71f7de201)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.012096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.145753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea4a9f1a3fa4892bbe2c2b71f7de201', 'request_id': 'req-7ddf7726-3589-4f4c-9c66-abaecb59134c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.152306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea4a9f1a3fa4892bbe2c2b71f7de201', 'request_id': 'req-7ddf7726-3589-4f4c-9c66-abaecb59134c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.273599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea4a9f1a3fa4892bbe2c2b71f7de201', 'request_id': 'req-7ddf7726-3589-4f4c-9c66-abaecb59134c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.285130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] The token cache is being invalidate because project 1ea4a9f1a3fa4892bbe2c2b71f7de201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:02.289522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.289522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7ddf7726-3589-4f4c-9c66-abaecb59134c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:02.289522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1083/2190] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:01 2022] DELETE /identity/v3/projects/1ea4a9f1a3fa4892bbe2c2b71f7de201 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:02.292563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.293263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.293263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.312893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:02.314639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.333932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:02.378910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1108/2191] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:02.386801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.387089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.387405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.408952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:02.410163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:02.418934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1084/2192] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:02.423599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.434548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828148da-6ad8-484b-ac6c-af92ad82eb56 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.439407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.440013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.440288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.440557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.441429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:37:02.441732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.444463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.458958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1109/2193] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:37:02.463544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.473689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669d073-9efd-495a-981d-05dfec89b357 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.478674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.479310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.479619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.479892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.480537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.482458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.595058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1085/2194] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:37:02.599807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.610015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0985e24-6686-4e36-8c98-da9d0af54e03 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.669419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.670031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.670311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.670584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.671258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.673178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.818724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1110/2195] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:37:02 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:37:02.822922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.835030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303ed1-b817-45a3-8af3-a022dcfea805 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.840245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.840894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.841173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.841445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.842097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.843809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.855237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1086/2196] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.859335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.872623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022331-35e1-46a8-a440-12b7898cb25b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.877851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.878460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.878731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.878993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.879680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.881270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b958a2-017e-461e-b3c3-d7a9f621c5f2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:02.892230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1111/2197] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:37:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:37:02.896959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:02.911173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473a990-556e-46ac-b8db-8b5e0ac5715a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:02.915899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:02.916564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:02.916840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:02.917112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] PATH_INFO: `/v3/projects/528a83a2aa38475d85643530fc611a7e/users/56745d112fd94e7085c1208b76edc30e/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:02.933803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=528a83a2aa38475d85643530fc611a7e, user_id=56745d112fd94e7085c1208b76edc30e, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:02.936391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.111052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1087/2198] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:37:02 2022] PUT /identity/v3/projects/528a83a2aa38475d85643530fc611a7e/users/56745d112fd94e7085c1208b76edc30e/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:03.122651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.122974 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.123248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.158511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] MFA Rules not processed for user `56745d112fd94e7085c1208b76edc30e`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:03.160003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.230863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1112/2199] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:03.243134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:03.245057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:03.270245 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:03.319274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da49bb-1c36-49cb-82b0-3064499d2dce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:03.326939 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:03.328007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:03.328493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:03.328968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:03.339667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:03.341291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9840d76-9c97-4e89-9732-01c19af6ed9f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:03.352216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1088/2200] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:37:13.227944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:13.456933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=17 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:29.444070 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:29.523062 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817a5fd-4f84-4160-bc6e-eb480fb037a4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:29.528261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:29.528932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:29.529260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:29.529548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] PATH_INFO: `/v3/users/56745d112fd94e7085c1208b76edc30e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:29.530790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56745d112fd94e7085c1208b76edc30e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:29.532791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:29.735793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56745d112fd94e7085c1208b76edc30e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:29.834523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56745d112fd94e7085c1208b76edc30e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:29.835734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56745d112fd94e7085c1208b76edc30e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:29.850971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56745d112fd94e7085c1208b76edc30e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:29.851782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56745d112fd94e7085c1208b76edc30e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:29.868497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1113/2201] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:37:29 2022] DELETE /identity/v3/users/56745d112fd94e7085c1208b76edc30e => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:37:29.873358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:29.874963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:29.874963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:29.894938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:29.896262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:29.959253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1089/2202] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:37:29.969324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:29.971379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:30.031429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdabf0f-2b85-4ab4-ab98-71226833c4c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:30.037032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:30.037644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:30.037969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:30.038276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:30.049287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:30.050835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f34d3b-8014-411a-b36e-1d11ffc113ae service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:30.061748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1114/2203] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:30.668884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:30.668884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:30.682252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2bf0f-d2f0-47c2-b79d-4f609d1f9109 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:30.687452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:30.688186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:30.688387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:30.688610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] PATH_INFO: `/v3/projects/528a83a2aa38475d85643530fc611a7e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:30.690861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] RBAC: Authorizing `identity:delete_project(project_id=528a83a2aa38475d85643530fc611a7e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:30.694098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:30.836361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528a83a2aa38475d85643530fc611a7e', 'request_id': 'req-4def3d95-60a0-4a2c-a960-1343fb4e473a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:30.843017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528a83a2aa38475d85643530fc611a7e', 'request_id': 'req-4def3d95-60a0-4a2c-a960-1343fb4e473a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:30.926128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528a83a2aa38475d85643530fc611a7e', 'request_id': 'req-4def3d95-60a0-4a2c-a960-1343fb4e473a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:30.937563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] The token cache is being invalidate because project 528a83a2aa38475d85643530fc611a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:37:30.937862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:30.938504 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:37:30.940113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1090/2204] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:37:30 2022] DELETE /identity/v3/projects/528a83a2aa38475d85643530fc611a7e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:37:39.628278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a302-95d8-4a26-b33e-8b881603c094 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:39.628906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a302-95d8-4a26-b33e-8b881603c094 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:39.628906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e607a302-95d8-4a26-b33e-8b881603c094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:39.648767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e607a302-95d8-4a26-b33e-8b881603c094 None None] MFA Rules not processed for user `8a45ee95cf5947c49584ff403facaff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:39.650026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e607a302-95d8-4a26-b33e-8b881603c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:39.670375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:39.717932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1115/2205] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:40.934135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:45.654032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:45.655704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:45.718033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def3d95-60a0-4a2c-a960-1343fb4e473a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:45.724044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:45.724044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:45.724044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:45.724044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:45.725866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:45.835987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:45.837733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:45.850672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1091/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:37:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:37:46.933593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:46.934479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:46.934919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:46.962508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:37:46.964039 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:47.034622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1116/2207] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:37:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:37:47.094350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:37:47.098509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:37:47.188496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69af0a-8b3a-403b-b05c-ec3e274458b1 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:37:47.194902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:37:47.195657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:37:47.196110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:37:47.196374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:37:47.207290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:37:47.212622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:37:47.220133 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1092/2208] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:37:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:37:57.078951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=40 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:37:57.215881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=82 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:04.714969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:04.729924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc0d27-2e61-491b-b7ad-4f766df3e853 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:04.733840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:04.734454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:04.734732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:04.735003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:04.737018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:04.822406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:04.829969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:04.835943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1117/2209] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:09.473023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:09.474424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:09.486344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d98f8-8f3f-4b3b-9982-37dbd767dc94 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:09.491735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:09.491735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:09.491735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:09.491735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:09.493105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:09.507296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:09.508862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:09.521677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1093/2210] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:11.839550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:11.841067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:11.856783 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1efc-ecea-404f-8e49-92d0c7715f7a service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:11.863330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:11.863864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:11.864141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:11.864415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:11.866211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:11.949074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:11.950960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:11.962570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1118/2211] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:12.656212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:12.657531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:12.671812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c16923-7f0f-46e9-8adc-c6324867cc86 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:12.678151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:12.678733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:12.679030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:12.679330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:12.681027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:12.755273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:12.757357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cbad1-958c-4199-8cde-1e750579b172 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:12.768162 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1094/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:14.568165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:14.581506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24db5a3-9f77-434b-83da-1c20c9958795 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:14.585629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:14.586254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:14.586773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:14.587016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:14.589592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:14.600128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:14.601587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f8997a-b508-407b-b38a-b2d79aa1044e service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:14.610504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1119/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:15.003934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.009660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.009660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.009660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1095/2214] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:38:15.013588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ed2c-27d3-4707-b7ef-57410079432a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.013936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ed2c-27d3-4707-b7ef-57410079432a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.014313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8ed2c-27d3-4707-b7ef-57410079432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.036484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-21f8ed2c-27d3-4707-b7ef-57410079432a None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:15.037793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-21f8ed2c-27d3-4707-b7ef-57410079432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.047470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1120/2215] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:15.059930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:15.061693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.128095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa20c19-a77e-459e-82f1-6a12a9724df2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:15.135078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:15.136125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.136568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.136974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.155247 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:15.157702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672219df-6170-4a0b-a58d-30bc883b3b63 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:15.174895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1096/2216] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:15.461094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e60e-2136-4cc1-940a-ecf7bdecd81b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.461382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e60e-2136-4cc1-940a-ecf7bdecd81b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.461667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e60e-2136-4cc1-940a-ecf7bdecd81b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.463013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1121/2217] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:38:15.468282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67d53-a749-4f71-9ef9-afcb43d172f2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.468574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67d53-a749-4f71-9ef9-afcb43d172f2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.468884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d67d53-a749-4f71-9ef9-afcb43d172f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.490052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e9d67d53-a749-4f71-9ef9-afcb43d172f2 None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:15.491369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9d67d53-a749-4f71-9ef9-afcb43d172f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.498262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1097/2218] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:15.511119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e60e-2136-4cc1-940a-ecf7bdecd81b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:15.523577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e60e-2136-4cc1-940a-ecf7bdecd81b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:15.527736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:15.529218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.529218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.529218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.537191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:15.538761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5ab6ee-ccdf-4725-b2b2-4dbff1c98ba6 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:15.546751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1122/2219] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:15.642148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2bfbf-5d12-44a8-98e9-841b79565de0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.642438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2bfbf-5d12-44a8-98e9-841b79565de0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.642701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2bfbf-5d12-44a8-98e9-841b79565de0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.645080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1098/2220] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:38:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:38:15.650585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.651048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.651355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.676950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:15.678324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.688040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1123/2221] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:15.698787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2bfbf-5d12-44a8-98e9-841b79565de0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:15.710822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2bfbf-5d12-44a8-98e9-841b79565de0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:15.714479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:15.715017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.715334 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.715600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.728237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:15.729709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef72e-3402-4640-93a4-dce16fa73fd0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:15.742257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1099/2222] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:38:15.811453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:15.814527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.869822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:15.894159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d05d-0f05-4ba9-8801-c0e6cd219278 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:15.899843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:15.900457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:15.901488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:15.901488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:15.902862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:15.915523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:15.917129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19286c7e-c334-489e-8813-9b8544202586 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:15.928542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1124/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:19.397672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:19.398187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:19.398187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:19.421473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] MFA Rules not processed for user `3a32dcea37fb4255823fec5eb5c57e00`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:19.422672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:19.484436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1100/2224] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:38:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:38:23.774758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:23.775317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:23.775571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:23.796805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:38:23.798066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:23.806539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1125/2225] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:38:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:38:23.819790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:23.833889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0ba8e-eced-4472-949c-2aa3ed2748c9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:23.839770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:23.840480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:23.840861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:23.841404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:23.851341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:23.853138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:23.858325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:23.863820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1101/2226] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:38:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:38:33.791246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=15 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:39.781139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:39.794428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6f41-d4ba-4936-93ed-6633e799ef34 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:39.797142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:39.797748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:39.798021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:39.798292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:39.800941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:39.811234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:39.812690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:39.820743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1126/2227] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:49.817150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:38:52.717567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:52.730014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b37f1c-eb48-4c95-9051-17b4b9c231d5 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:52.735681 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:52.736299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:52.736580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:52.736853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:52.739299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:52.817193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:52.818856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd1221-d6f5-4509-8f35-247b22e3b38a service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:52.828781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1102/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:38:55.862341 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:38:55.876555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c9586-3399-4799-84a8-b2581a50b9e4 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:38:55.883180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:38:55.883797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:38:55.884077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:38:55.884351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:38:55.886093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:38:55.899599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:38:55.901206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:38:55.913530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1127/2229] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:02.243367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c5d65-3672-4f2f-b855-1cd0d610b40b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.243876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c5d65-3672-4f2f-b855-1cd0d610b40b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.243995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c5d65-3672-4f2f-b855-1cd0d610b40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.264465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-969c5d65-3672-4f2f-b855-1cd0d610b40b None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:02.265679 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-969c5d65-3672-4f2f-b855-1cd0d610b40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:02.273438 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1103/2230] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:02.296453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:02.308614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95558ffa-e4a0-43f5-8e5d-a6feac399d58 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:02.313872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:02.314501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:02.314773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:02.315038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:02.324238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:02.325816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:02.343894 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1128/2231] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:39:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:39:12.258785 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=31 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:12.331053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=12 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:33.480155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f357-1f27-44df-9111-b815478e555f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:33.480668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f357-1f27-44df-9111-b815478e555f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:33.480765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982f357-1f27-44df-9111-b815478e555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:33.500985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3982f357-1f27-44df-9111-b815478e555f None None] MFA Rules not processed for user `8937e9f80f1540e389ee283971624741`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:39:33.505050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3982f357-1f27-44df-9111-b815478e555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:33.509773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1104/2232] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:39:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:39:40.328644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:40.343200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ca4ff9-7291-4737-92e9-ccb8a057561e service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:40.348816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:40.349440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:40.349723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:40.349995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:40.351806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:40.364396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:40.366698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:40.378380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1129/2233] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:42.627703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982f357-1f27-44df-9111-b815478e555f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:39:42.649844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982f357-1f27-44df-9111-b815478e555f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:39:42.663766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:39:42.664812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:39:42.665667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:39:42.665667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:39:42.677487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:39:42.701121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:39:42.703267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:39:42.717031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1105/2234] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:39:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:39:50.373123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:39:52.711135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:14.583644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:14.595478 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012b2db-8eda-4234-9884-9a1ec43cbd6a service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:14.602785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:14.602785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:14.602785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:14.603183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:14.604709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:14.618690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:14.620431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:14.630993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1130/2235] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:14.741905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:14.744328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:14.811684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c0546b-29be-4a0e-99eb-b4b7ab518c61 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:14.816227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:14.817237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:14.817237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:14.817407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] PATH_INFO: `/v3/users/3a32dcea37fb4255823fec5eb5c57e00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:14.818534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a32dcea37fb4255823fec5eb5c57e00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:14.821156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:15.008498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a32dcea37fb4255823fec5eb5c57e00'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.009177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a32dcea37fb4255823fec5eb5c57e00'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.079221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a32dcea37fb4255823fec5eb5c57e00'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.087974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a32dcea37fb4255823fec5eb5c57e00'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.100550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a32dcea37fb4255823fec5eb5c57e00'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.107396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1106/2236] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:40:14 2022] DELETE /identity/v3/users/3a32dcea37fb4255823fec5eb5c57e00 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:15.475717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:15.495740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf47277-4866-490d-a729-f8ce968c8601 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:15.558772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:15.559450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:15.559731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:15.560005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] PATH_INFO: `/v3/projects/98240e01aaa84f3e8c7e425ad2c9eb4f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:15.561279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=98240e01aaa84f3e8c7e425ad2c9eb4f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:15.563267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:15.674475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98240e01aaa84f3e8c7e425ad2c9eb4f', 'request_id': 'req-cef33c89-88ae-42e0-87d6-483235a2ea0a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.681708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98240e01aaa84f3e8c7e425ad2c9eb4f', 'request_id': 'req-cef33c89-88ae-42e0-87d6-483235a2ea0a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.774915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98240e01aaa84f3e8c7e425ad2c9eb4f', 'request_id': 'req-cef33c89-88ae-42e0-87d6-483235a2ea0a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.786082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] The token cache is being invalidate because project 98240e01aaa84f3e8c7e425ad2c9eb4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:40:15.786795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.787071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cef33c89-88ae-42e0-87d6-483235a2ea0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:40:15.788592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1131/2237] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:40:15 2022] DELETE /identity/v3/projects/98240e01aaa84f3e8c7e425ad2c9eb4f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:40:20.187558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:40:20.189018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:20.269920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e7c3b-8a36-4f6c-81c9-46404c06dd53 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:40:20.283180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:20.337829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:40:20.338579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:20.338853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:20.339120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:20.340940 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:20.403620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:40:20.405280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f1f31-6497-40fe-a649-994790e368c4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:40:20.415204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1107/2238] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:40:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:40:25.782850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:30.410296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=48 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:37.651105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:37.651705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:37.652231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:37.673391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] MFA Rules not processed for user `8a45ee95cf5947c49584ff403facaff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:37.674704 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:37.685444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1132/2239] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:47.362352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8c9b-1fde-46d2-9c77-7b63ca3a5b9e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:40:47.362921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8c9b-1fde-46d2-9c77-7b63ca3a5b9e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:40:47.362921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe8c9b-1fde-46d2-9c77-7b63ca3a5b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:40:47.384222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dabe8c9b-1fde-46d2-9c77-7b63ca3a5b9e None None] MFA Rules not processed for user `ab4df1724851433889a519583a5333b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:40:47.385352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dabe8c9b-1fde-46d2-9c77-7b63ca3a5b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:40:47.444223 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1108/2240] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:40:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:40:47.666778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:40:57.443576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=25 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:11.019893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:11.022089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:11.169322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9982ea9-c03e-49e8-83ec-c78bdd1fe176 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:11.177390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:11.178028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:11.178393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:11.178737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:11.181269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:11.281081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:11.286316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:11.302350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1133/2241] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:14.067683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.068308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.068308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.098118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:14.099352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:14.155893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1109/2242] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:41:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:14.297150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:14.298553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:14.370984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd0747-6fae-4f58-8bad-6ab7a5086e52 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:14.376097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:14.376705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:14.376982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:14.377249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:14.386246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:14.387891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:14.396642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1134/2243] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:41:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:41:15.483051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:15.485014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:15.566848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f96dc8-4b5e-421b-89dd-bf4ee9ddbba0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:15.572096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:15.572838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:15.573122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:15.573398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:15.575299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:15.662244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:15.664387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d0791a-c0b9-4e0e-8f30-77c2f13aa31d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:15.678706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:15.684460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1110/2244] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:18.518652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:18.521052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:18.543032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a614c-d75a-4f22-9593-8e906f20cf59 service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:18.547929 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:18.548540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:18.548816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:18.549086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:18.551031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:18.565088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:18.566740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9abf16-a717-40e0-b500-f2e836ad3e8a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:18.573575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:18.578948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1135/2245] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:22.714864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:22.715461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:22.715560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:22.717029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1111/2246] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:41:22.747380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:22.747949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:22.747949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:22.816598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] MFA Rules not processed for user `0688b8d8c0aa46818633fbc17dfb58d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:22.818056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:22.864380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:22.866332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:22.890065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68018b7f-52c7-4ce0-bdf6-46f872e26efa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:22.895283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:22.896163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:22.896665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:22.897268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] PATH_INFO: `/v3/users/8937e9f80f1540e389ee283971624741` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:22.898620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8937e9f80f1540e389ee283971624741)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:22.900779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:22.914233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1136/2247] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:41:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:22.935257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:22.937233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:22.964090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffebf7-a1dd-4c97-8ec6-e3e14f0a2e02 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:22.975882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:22.977672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:22.977672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:22.978016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:22.987838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:22.989444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:23.004449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1137/2248] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:23.116492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8937e9f80f1540e389ee283971624741'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.117539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8937e9f80f1540e389ee283971624741'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.222642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8937e9f80f1540e389ee283971624741'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.232124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8937e9f80f1540e389ee283971624741'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.263836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01e5f278-aa5e-4e99-aa54-f0b54c57afa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8937e9f80f1540e389ee283971624741'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.269324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1112/2249] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:22 2022] DELETE /identity/v3/users/8937e9f80f1540e389ee283971624741 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:23.629131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:23.654543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be8a03-1737-4b2a-abc2-d047e34dea55 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:23.741886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:23.742597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:23.743373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:23.744338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] PATH_INFO: `/v3/projects/2f0958b1518e4439bf12959b7760abc4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:23.745598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0958b1518e4439bf12959b7760abc4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:23.747975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:23.866188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:23.873675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0958b1518e4439bf12959b7760abc4', 'request_id': 'req-d0b59e1b-3b72-4d77-9da3-3ee803d62281'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.888493 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0958b1518e4439bf12959b7760abc4', 'request_id': 'req-d0b59e1b-3b72-4d77-9da3-3ee803d62281'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:23.996533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0958b1518e4439bf12959b7760abc4', 'request_id': 'req-d0b59e1b-3b72-4d77-9da3-3ee803d62281'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.009291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] The token cache is being invalidate because project 2f0958b1518e4439bf12959b7760abc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:24.009781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.010574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0b59e1b-3b72-4d77-9da3-3ee803d62281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.012087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1138/2250] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:23 2022] DELETE /identity/v3/projects/2f0958b1518e4439bf12959b7760abc4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:24.017173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.017674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.017792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.069978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:24.072241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.143447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1113/2251] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.152296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.152662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.152941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.173946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:24.175197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.184290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1139/2252] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.188746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.198739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa5bcf-ee96-4bc1-bddd-b477bd1181c2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.203771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.204382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.204658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.204928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.205579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:24.205898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.207540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.226475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1114/2253] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:24.231467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.240884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ba40d-544c-4e3d-a708-436a4960ba97 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.246741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.247317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.247593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.247860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.248520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.250421 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.286576 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.288687 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.304025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500afe9-21e0-4a76-8e31-bfa1900ebe4e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.309781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.311331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.311331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.311331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] PATH_INFO: `/v3/users/cf9862b97ba64cc09b4903ba24ac2181` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.312109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9862b97ba64cc09b4903ba24ac2181)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.314186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.341356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1140/2254] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:24.345784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.411946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a98598-1ec6-42e6-863c-1935643ad35b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.417228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.417826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.418099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.418367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.418996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.420989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.486213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:24.501515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9862b97ba64cc09b4903ba24ac2181'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.502188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9862b97ba64cc09b4903ba24ac2181'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.526822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1141/2255] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:24.531130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.532865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.544730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e93ede-bcee-4faf-bba2-5338a69b99fc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.549444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.550019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.550293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.550558 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.551216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.552847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.563664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1142/2256] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:24.576648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.586636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79fde0-37b9-465f-bd13-4381e8c99b5f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.592069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.592696 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.592967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.593237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.593814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9862b97ba64cc09b4903ba24ac2181'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.593969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.595471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.601541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9862b97ba64cc09b4903ba24ac2181'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.610031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1143/2257] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:24.611640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.615378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-edac6f32-c1fa-457d-aa46-f10c4c7f6816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9862b97ba64cc09b4903ba24ac2181'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:24.622295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1115/2258] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:24 2022] DELETE /identity/v3/users/cf9862b97ba64cc09b4903ba24ac2181 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:24.622481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274701f-3782-438d-9e74-a60c4f84adb4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.697797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.698574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.699257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.699371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] PATH_INFO: `/v3/projects/389e5ef34dcd47728ad0e57d77364f1e/users/fd6962beac3345d1901e977712c99ac8/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.717943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389e5ef34dcd47728ad0e57d77364f1e, user_id=fd6962beac3345d1901e977712c99ac8, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.720272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:24.722735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:24.829792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1144/2259] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:24 2022] PUT /identity/v3/projects/389e5ef34dcd47728ad0e57d77364f1e/users/fd6962beac3345d1901e977712c99ac8/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:24.840799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.841112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.841388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.870362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] MFA Rules not processed for user `fd6962beac3345d1901e977712c99ac8`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:24.871659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.876592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.878364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.954942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1116/2260] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:24.963375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00660e4f-8c9c-470d-8171-df57d82bd4c9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:24.968136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:24.968318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:24.968868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:24.969155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:24.969416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] PATH_INFO: `/v3/projects/9193c40d930d4774b800ddfeefbf8c92` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:24.969522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:24.970548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9193c40d930d4774b800ddfeefbf8c92)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:24.972610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:25.045431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6bd7d-bebe-4369-b580-b39a9565b87d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:25.051019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:25.051651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:25.051925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:25.052227 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:25.061155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:25.062818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:25.071964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1117/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:25.101382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9193c40d930d4774b800ddfeefbf8c92', 'request_id': 'req-1d84234c-6c20-4be3-a494-5f04f011c74e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.107772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9193c40d930d4774b800ddfeefbf8c92', 'request_id': 'req-1d84234c-6c20-4be3-a494-5f04f011c74e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.207385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9193c40d930d4774b800ddfeefbf8c92', 'request_id': 'req-1d84234c-6c20-4be3-a494-5f04f011c74e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.219623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] The token cache is being invalidate because project 9193c40d930d4774b800ddfeefbf8c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:25.219908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.220563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.222118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1145/2262] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:24 2022] DELETE /identity/v3/projects/9193c40d930d4774b800ddfeefbf8c92 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:41:25.226142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:25.237243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01ddff-a82c-4b49-a959-0683ac979186 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:25.256520 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:25.297432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:25.298066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:25.298355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:25.298608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] PATH_INFO: `/v3/users/086c17f07797436b85b8dd6615112aaf` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:25.299743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=086c17f07797436b85b8dd6615112aaf)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:25.301672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:25.500905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '086c17f07797436b85b8dd6615112aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.501496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '086c17f07797436b85b8dd6615112aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.574101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '086c17f07797436b85b8dd6615112aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.581325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '086c17f07797436b85b8dd6615112aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.593618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '086c17f07797436b85b8dd6615112aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:25.599535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1118/2263] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:25 2022] DELETE /identity/v3/users/086c17f07797436b85b8dd6615112aaf => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:25.850989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:25.853011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:25.867823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84234c-6c20-4be3-a494-5f04f011c74e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:25.928581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:25.929292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:25.929643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:25.929915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] PATH_INFO: `/v3/projects/df1b6a3f534f4ec682f5f73c1292124f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:25.932017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=df1b6a3f534f4ec682f5f73c1292124f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:25.934001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:26.060927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:26.076995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df1b6a3f534f4ec682f5f73c1292124f', 'request_id': 'req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:26.083036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df1b6a3f534f4ec682f5f73c1292124f', 'request_id': 'req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:26.189487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df1b6a3f534f4ec682f5f73c1292124f', 'request_id': 'req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:26.200362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] The token cache is being invalidate because project df1b6a3f534f4ec682f5f73c1292124f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:26.200707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:26.201394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c85bd9e-6bc5-4a5d-8bde-1c65e5c7dd4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:26.202906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1146/2264] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:25 2022] DELETE /identity/v3/projects/df1b6a3f534f4ec682f5f73c1292124f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:35.682500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:36.269857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:36.496531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.496531 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.508642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c2507-dd6a-4077-996f-c14cd0d4d6e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:36.565005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:36.565635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.565915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.566190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] PATH_INFO: `/v3/users/fd6962beac3345d1901e977712c99ac8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.567340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd6962beac3345d1901e977712c99ac8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:36.569297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:36.799386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd6962beac3345d1901e977712c99ac8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.800070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd6962beac3345d1901e977712c99ac8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.847358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd6962beac3345d1901e977712c99ac8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.854704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd6962beac3345d1901e977712c99ac8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.866364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd6962beac3345d1901e977712c99ac8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:36.872806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1119/2265] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:41:36 2022] DELETE /identity/v3/users/fd6962beac3345d1901e977712c99ac8 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:36.877348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:36.877622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:36.877895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:36.897126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:36.898326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:36.958569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1147/2266] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:36.969100 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:36.973171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:37.028853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb8a3d-ad5e-472a-aa0a-f384a93ddb4f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:37.032455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:37.033058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.033336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.033608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.042809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:37.044279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e40bf-f8b7-4cd3-8aeb-0cafbfd35a3b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:37.053625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1120/2267] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:37.477661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:37.479202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:37.494169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea5023-686f-456e-a8a1-71af48b2f549 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:37.499680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:37.500380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.500664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.500937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] PATH_INFO: `/v3/projects/389e5ef34dcd47728ad0e57d77364f1e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.502075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=389e5ef34dcd47728ad0e57d77364f1e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:37.504082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:37.613748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389e5ef34dcd47728ad0e57d77364f1e', 'request_id': 'req-e026da2e-a53f-4aad-8cb1-61a68d808a8a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:37.620745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389e5ef34dcd47728ad0e57d77364f1e', 'request_id': 'req-e026da2e-a53f-4aad-8cb1-61a68d808a8a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:37.704417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389e5ef34dcd47728ad0e57d77364f1e', 'request_id': 'req-e026da2e-a53f-4aad-8cb1-61a68d808a8a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:37.717414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] The token cache is being invalidate because project 389e5ef34dcd47728ad0e57d77364f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:41:37.717705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:37.718381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e026da2e-a53f-4aad-8cb1-61a68d808a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:41:37.719916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1148/2268] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:41:37 2022] DELETE /identity/v3/projects/389e5ef34dcd47728ad0e57d77364f1e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:37.724916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.725228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.725502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.746158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:37.747389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:37.767415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:37.818333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1121/2269] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:37.826641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.826928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.827243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.847131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:37.848296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:37.856886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1149/2270] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:37.861237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:37.871560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c652eb9-ccf2-4779-befc-2fde8fb1302e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:37.876341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:37.876937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.877213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.877512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.878152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:41:37.878452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:37.880090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:37.891833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1122/2271] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:41:37.895986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:37.905159 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d90cab-746c-4b83-9ba8-098b2bcff926 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:37.909890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:37.910481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:37.910761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:37.911038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:37.911716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:37.913617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.007268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1150/2272] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:37 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:38.011945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.022054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32a35e-5304-464a-a991-40a88ead3e80 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.078564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.079192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.079473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.079744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.080368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.082345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.189773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1123/2273] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:38 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:38.193757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.204292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b942f2-7637-4d4c-a501-78f6a5a8a46b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.208831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.209812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.209812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.210096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.210938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.213396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.223626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1151/2274] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:38.228615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.238670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c4253-8bc6-42cd-b527-8371dd8658aa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.244140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.244770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.245050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.245320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.245975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.248108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7ed40-b8b6-4323-a352-d82cdf5aea6a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.259236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1124/2275] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:38.263221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.273137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5806-2334-414e-a308-9b6732ba5072 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.278496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.279114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.279574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.279852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] PATH_INFO: `/v3/projects/9ecffb49f2c4481ab6b58eb1622990ef/users/659ceefb0ea14f7d8c9396bc6e4a4719/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.294808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ecffb49f2c4481ab6b58eb1622990ef, user_id=659ceefb0ea14f7d8c9396bc6e4a4719, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.297495 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.388610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1152/2276] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:38 2022] PUT /identity/v3/projects/9ecffb49f2c4481ab6b58eb1622990ef/users/659ceefb0ea14f7d8c9396bc6e4a4719/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:38.401004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.401279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.401546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.429214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] MFA Rules not processed for user `659ceefb0ea14f7d8c9396bc6e4a4719`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:38.430419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:38.495577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1125/2277] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:38.500172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.521383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:38.561680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea53061-2e7e-4538-9927-1fce50af210c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.566849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.567492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.567754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.568030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.568663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.570624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.653169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1153/2278] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:41:38 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:41:38.658175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.669472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6bd190-1dfc-4eed-be04-c846d9590a26 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.681992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:38.726875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.727526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.727951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.728403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.729044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.731035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.800350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1126/2279] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:41:38 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:41:38.804371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.815917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21096-3c85-41d8-8e87-5cd7f6c08668 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.820632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.821213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.822071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.822417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.823198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.824791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.835297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1154/2280] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:41:38.848477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.858700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85960b78-c99d-4511-aad0-34155802312d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:38.863399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:38.864060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:38.864331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:38.864602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] PATH_INFO: `/v3/projects/2ddddc206db64e31a69fcec31d650360/users/da22f1fceeb44a3fa7ff2c891e55c3df/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:38.880567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ddddc206db64e31a69fcec31d650360, user_id=da22f1fceeb44a3fa7ff2c891e55c3df, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:38.883257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:38.944300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1127/2281] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:38 2022] PUT /identity/v3/projects/2ddddc206db64e31a69fcec31d650360/users/da22f1fceeb44a3fa7ff2c891e55c3df/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:38.948985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:38.959021 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf3f34-8da3-4acd-8d52-e5c247617a3b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.017167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.017747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.018018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.018391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.019024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.020638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba0c5b-19a4-49af-84bf-3464ed16cf8b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.030591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1155/2282] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:41:39.034833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.046723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e335a-1730-415e-b958-543add27a155 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.051349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.051932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.052277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.052474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] PATH_INFO: `/v3/projects/2ddddc206db64e31a69fcec31d650360/users/da22f1fceeb44a3fa7ff2c891e55c3df/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.054356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ddddc206db64e31a69fcec31d650360, user_id=da22f1fceeb44a3fa7ff2c891e55c3df, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.057784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.067987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1128/2283] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:41:39 2022] PUT /identity/v3/projects/2ddddc206db64e31a69fcec31d650360/users/da22f1fceeb44a3fa7ff2c891e55c3df/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:41:39.078952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.079271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.079546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.107652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] MFA Rules not processed for user `da22f1fceeb44a3fa7ff2c891e55c3df`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:39.108848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.176192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1156/2284] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:39.188000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.189832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:39.269252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ba9b4-8091-4016-a310-0f82b845ef34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.272994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.273598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.273874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.274143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.283523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.284965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.295076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1129/2285] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:39.468044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:39.478102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8baba2-9937-4752-9505-3224d8e63437 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:39.481952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:39.482588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:39.482867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:39.483162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:39.496071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:39.544504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:39.545942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c16d8-028a-4429-b134-6466ac58cb5b service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:39.555255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1157/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:49.290152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:49.579049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=21 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:41:52.897757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:52.899230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:52.973770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b988f-9fc2-4484-a543-b4d7bd800148 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:52.982582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:52.983393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:52.983701 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:52.983937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:52.986074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:53.001524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:53.003558 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:53.017331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1130/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:41:57.609270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:57.610049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:57.610049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:57.633057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] MFA Rules not processed for user `21e6ff7908e84a5994a51be0cfc82414`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:41:57.634436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:57.691551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1158/2288] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:41:58.019184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:41:58.021892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.088789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f882f92-0d52-4518-91c6-02289a7764a0 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:41:58.092624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:41:58.093235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:41:58.093516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:41:58.093786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:41:58.095577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:41:58.105166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:41:58.106603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b061b-8c9a-4f9e-8a94-33b08a2e0083 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:41:58.114429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1131/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:07.684004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=25 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:08.110906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=58 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:38.292676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:38.354945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aac2d9-b4d7-4046-99e5-f8b213a1f177 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:38.359636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:38.360197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:38.360479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:38.360823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] PATH_INFO: `/v3/users/659ceefb0ea14f7d8c9396bc6e4a4719` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:38.361977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659ceefb0ea14f7d8c9396bc6e4a4719)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:38.363942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:38.543948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659ceefb0ea14f7d8c9396bc6e4a4719'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:38.597716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659ceefb0ea14f7d8c9396bc6e4a4719'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:38.598339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659ceefb0ea14f7d8c9396bc6e4a4719'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:38.610945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659ceefb0ea14f7d8c9396bc6e4a4719'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:38.611705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659ceefb0ea14f7d8c9396bc6e4a4719'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:38.624307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1159/2290] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:38 2022] DELETE /identity/v3/users/659ceefb0ea14f7d8c9396bc6e4a4719 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:38.629399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:38.629905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:38.630004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:38.649894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:38.651093 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:38.711524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1132/2291] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:38.722241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:38.723665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:38.782414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3baca5-fdc8-4215-9194-f0b0255ae7f3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:38.786926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:38.787574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:38.787848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:38.788117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:38.797718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:38.799251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:38.808673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1160/2292] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:42:39.238757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:39.245842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:39.260144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421043d-8815-48c9-a0d5-12558884b9fc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:39.269375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:39.270033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:39.270311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:39.270651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] PATH_INFO: `/v3/projects/9ecffb49f2c4481ab6b58eb1622990ef` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:39.271720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ecffb49f2c4481ab6b58eb1622990ef)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:39.273628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:39.386171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ecffb49f2c4481ab6b58eb1622990ef', 'request_id': 'req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.393388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ecffb49f2c4481ab6b58eb1622990ef', 'request_id': 'req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.466226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ecffb49f2c4481ab6b58eb1622990ef', 'request_id': 'req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.485319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] The token cache is being invalidate because project 9ecffb49f2c4481ab6b58eb1622990ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:39.485319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.485319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.485319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1133/2293] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:39 2022] DELETE /identity/v3/projects/9ecffb49f2c4481ab6b58eb1622990ef => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:39.488651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:39.499102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aa690-9bcb-4cea-b82f-bfafab7f12ce service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:39.518721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:39.558867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:39.559514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:39.559788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:39.560052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] PATH_INFO: `/v3/users/da22f1fceeb44a3fa7ff2c891e55c3df` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:39.561201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da22f1fceeb44a3fa7ff2c891e55c3df)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:39.563525 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:39.759103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da22f1fceeb44a3fa7ff2c891e55c3df'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.843176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da22f1fceeb44a3fa7ff2c891e55c3df'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.843985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da22f1fceeb44a3fa7ff2c891e55c3df'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.858723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da22f1fceeb44a3fa7ff2c891e55c3df'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.859496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5e744069-d507-41ef-8b51-49ca17701915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da22f1fceeb44a3fa7ff2c891e55c3df'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:39.873546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1161/2294] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:39 2022] DELETE /identity/v3/users/da22f1fceeb44a3fa7ff2c891e55c3df => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:40.272810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:40.273822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:40.293428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8cd22b-9926-40b4-9bd9-4a987c68e8ca admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:40.355112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:40.355772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.356051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.356322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] PATH_INFO: `/v3/projects/2ddddc206db64e31a69fcec31d650360` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.357440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ddddc206db64e31a69fcec31d650360)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:40.359392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:40.440382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:40.458222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ddddc206db64e31a69fcec31d650360', 'request_id': 'req-a4892f60-d3f0-41ee-a02a-13e49e884f5c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:40.464708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ddddc206db64e31a69fcec31d650360', 'request_id': 'req-a4892f60-d3f0-41ee-a02a-13e49e884f5c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:40.586719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ddddc206db64e31a69fcec31d650360', 'request_id': 'req-a4892f60-d3f0-41ee-a02a-13e49e884f5c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:40.598182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] The token cache is being invalidate because project 2ddddc206db64e31a69fcec31d650360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:40.598478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:40.599196 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a4892f60-d3f0-41ee-a02a-13e49e884f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:40.601060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1134/2295] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:40 2022] DELETE /identity/v3/projects/2ddddc206db64e31a69fcec31d650360 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:40.605993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.609175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.609175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.627499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:40.628924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:40.698564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1162/2296] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:40.707019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.707293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.712766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.727999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:40.729200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:40.737728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1135/2297] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:40.742230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:40.752609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91f63e-bcfc-4eae-8f29-c4d4ac7ea398 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:40.757160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:40.757862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.758134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.758401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.759042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:40.759364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:40.760957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:40.772296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1163/2298] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:40.776611 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:40.786424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5344f7-f1fa-408d-bada-3cd57373f8cd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:40.791249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:40.791839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.792113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.792378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.793005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:40.794904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:40.914634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1136/2299] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:40 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:40.918913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:40.933753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc258f75-6599-401b-b7a7-365b391f999f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:40.960344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:40.993076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:40.993676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:40.993953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:40.994223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:40.994867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:40.996820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.102340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1164/2300] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:40 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:41.107168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.118585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5fef38-efb1-41b1-8dcf-e8887698e6bd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.123276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.127464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.127464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.127464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.127464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.127464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.137252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1137/2301] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:41.141237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.151521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa0ec-0e05-409b-bd8e-a8863cd087f8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.156174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.156771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.157043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.157308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.158732 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.160393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabe09e-a6c0-4d89-bc04-6cf1fdb8d424 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.170191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1165/2302] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:41.175201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.184934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308cfb3-e2c3-4c06-8d61-d452ed920056 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.189976 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.190613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.190898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.191218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] PATH_INFO: `/v3/projects/a37588c916364e0b91a75da6f4787b2d/users/153e1e056b804bcda35d01af4c999586/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.206437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37588c916364e0b91a75da6f4787b2d, user_id=153e1e056b804bcda35d01af4c999586, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.209107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.278323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1138/2303] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:41 2022] PUT /identity/v3/projects/a37588c916364e0b91a75da6f4787b2d/users/153e1e056b804bcda35d01af4c999586/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:42:41.292802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.293211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.293605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.328250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] MFA Rules not processed for user `153e1e056b804bcda35d01af4c999586`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:41.329640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:41.393705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1166/2304] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:41.398714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.462151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16a4e-c0b2-4b2a-ae10-d16e1a36d857 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.467057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.467691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.467970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.468240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.468882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.471490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.554091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1139/2305] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:41 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:41.558271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.569283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a3bed-83e8-430f-ba87-329fd2b0d6fd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.625368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.626082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.626295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.626671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.627346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.629265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.713241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1167/2306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:41 2022] POST /identity/v3/users => generated 299 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:41.718151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.729448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f5751-bfcd-496a-bbf3-ada9378b9829 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.736315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.736315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.736315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.736315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.736778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.737454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.748120 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1140/2307] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:41.752458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.762792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04a65-abca-461a-996c-385f0b420802 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.768074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.768714 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.768990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.769261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] PATH_INFO: `/v3/projects/5ba79a04576f4d41b796c5d2da522875/users/0e1dcddba9584aac8b282e32d6deadb6/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.784360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba79a04576f4d41b796c5d2da522875, user_id=0e1dcddba9584aac8b282e32d6deadb6, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.787019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.872687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1168/2308] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:41 2022] PUT /identity/v3/projects/5ba79a04576f4d41b796c5d2da522875/users/0e1dcddba9584aac8b282e32d6deadb6/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:41.878017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.888508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658ec73-4b17-4202-8333-0c87a53636e7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.911331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:41.950103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.950948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.951406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.952057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.952759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.954973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f269c4-6135-4a95-b011-62b440bb9e73 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:41.965346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1141/2309] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:41.969771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:41.975330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:41.983203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec900d-d747-49ac-ba16-3306ef29c464 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:41.986487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:41.987177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:41.987701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:41.987971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] PATH_INFO: `/v3/projects/5ba79a04576f4d41b796c5d2da522875/users/0e1dcddba9584aac8b282e32d6deadb6/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:41.989977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba79a04576f4d41b796c5d2da522875, user_id=0e1dcddba9584aac8b282e32d6deadb6, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:41.992658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.004102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1169/2310] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:41 2022] PUT /identity/v3/projects/5ba79a04576f4d41b796c5d2da522875/users/0e1dcddba9584aac8b282e32d6deadb6/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:42.015485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.015770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.016039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.045741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] MFA Rules not processed for user `0e1dcddba9584aac8b282e32d6deadb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:42.046959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.130684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1142/2311] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:42.143575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.145661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.204064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae1c1-0e88-4b89-866f-c645b2f9cf5e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.208781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.209376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.209655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.209921 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.220944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.222458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea89cab-5c02-4691-8b0c-327232264120 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.232949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1170/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:42.268343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.333559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323db09-cd78-4bca-8506-8be4559fff0e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.338361 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.339315 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.339592 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.339867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] PATH_INFO: `/v3/users/153e1e056b804bcda35d01af4c999586` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.341031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=153e1e056b804bcda35d01af4c999586)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.342991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.571714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '153e1e056b804bcda35d01af4c999586'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.572519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '153e1e056b804bcda35d01af4c999586'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.574290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:42.648189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '153e1e056b804bcda35d01af4c999586'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.663225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '153e1e056b804bcda35d01af4c999586'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.683113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '153e1e056b804bcda35d01af4c999586'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:42.690205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1143/2313] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:42 2022] DELETE /identity/v3/users/153e1e056b804bcda35d01af4c999586 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:42.696125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.696618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.697170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.718448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:42.719850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.780874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1171/2314] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:42.790831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:42.792295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:42.853735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451e9ea-d1c5-4295-9652-64f184d94b80 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:42.857720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:42.858384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:42.858704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:42.859016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:42.869560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:42.871225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:42.882883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1144/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:43.554451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:43.556047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:43.568586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1270990-3267-40f8-9b5e-2d5134aaba9b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:43.573329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:43.575420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:43.575420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:43.575420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] PATH_INFO: `/v3/projects/a37588c916364e0b91a75da6f4787b2d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:43.575761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37588c916364e0b91a75da6f4787b2d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:43.577496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:43.678769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37588c916364e0b91a75da6f4787b2d', 'request_id': 'req-b8958222-54fd-4864-ade6-0bf0589ba5e8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:43.684901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37588c916364e0b91a75da6f4787b2d', 'request_id': 'req-b8958222-54fd-4864-ade6-0bf0589ba5e8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:43.761733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37588c916364e0b91a75da6f4787b2d', 'request_id': 'req-b8958222-54fd-4864-ade6-0bf0589ba5e8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:43.773777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] The token cache is being invalidate because project a37588c916364e0b91a75da6f4787b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:43.777261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:43.777261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:43.777261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1172/2316] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:43 2022] DELETE /identity/v3/projects/a37588c916364e0b91a75da6f4787b2d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:43.780583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:43.790953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63d72-2a3d-4e92-9ba5-40430837fd56 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:43.848715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:43.849470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:43.849802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:43.850126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] PATH_INFO: `/v3/users/0e1dcddba9584aac8b282e32d6deadb6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:43.851379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e1dcddba9584aac8b282e32d6deadb6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:43.853759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:44.105575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e1dcddba9584aac8b282e32d6deadb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.106365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e1dcddba9584aac8b282e32d6deadb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.190954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e1dcddba9584aac8b282e32d6deadb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.199450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e1dcddba9584aac8b282e32d6deadb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.205364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:44.211983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e2683beb-d490-4aa9-9861-14ebeb8b4161 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e1dcddba9584aac8b282e32d6deadb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.218054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1145/2317] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:43 2022] DELETE /identity/v3/users/0e1dcddba9584aac8b282e32d6deadb6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:44.647750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:44.649212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:44.665012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8958222-54fd-4864-ade6-0bf0589ba5e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:44.678579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:44.738774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:44.739544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:44.739800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:44.740057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] PATH_INFO: `/v3/projects/5ba79a04576f4d41b796c5d2da522875` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:44.741237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba79a04576f4d41b796c5d2da522875)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:44.743199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:44.849779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba79a04576f4d41b796c5d2da522875', 'request_id': 'req-6aaaafb0-063c-45f4-abae-f0275722360f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.856314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba79a04576f4d41b796c5d2da522875', 'request_id': 'req-6aaaafb0-063c-45f4-abae-f0275722360f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.957368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba79a04576f4d41b796c5d2da522875', 'request_id': 'req-6aaaafb0-063c-45f4-abae-f0275722360f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.968618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] The token cache is being invalidate because project 5ba79a04576f4d41b796c5d2da522875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:44.968899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.969533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aaaafb0-063c-45f4-abae-f0275722360f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:44.971130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1173/2318] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:44 2022] DELETE /identity/v3/projects/5ba79a04576f4d41b796c5d2da522875 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:44.978086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:44.978623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:44.978943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:44.999049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:45.000308 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:45.067841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1146/2319] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:45.076103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.076390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.076661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.098135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:45.099417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:45.107480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1174/2320] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:45.112274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.122350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797125f-0e71-43ae-adce-f3de7e7dd807 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.128230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.128834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.129151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.129426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.130073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:45.130375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.132054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.144399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1147/2321] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:42:45.148906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.158379 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03be29d-9699-4b1e-be1a-f2def8c7b476 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.163071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.163710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.163986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.164255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.164896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.166809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.261000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1175/2322] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:45 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:45.265746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.275996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6982513-45b6-420b-b28f-3190ae1f626e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.336977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.338180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.338668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.339102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.340197 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.342181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.426484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1148/2323] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:45 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:45.430715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.442082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f40622-cd3c-45c1-b6ee-2560623e1439 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.446836 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.447577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.447889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.448215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.448810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.450404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.461764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1176/2324] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:45.466409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.477165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6aa25b-ec88-473a-aabe-005dbaf9250a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.482267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.483077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.483402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.483745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.484748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.486956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d60f7-d4f8-4ff7-a2d1-040e931d11dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.506172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1149/2325] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:45.511266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.526898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e360870-1dd3-4b72-bb44-36f8c54c92e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.533436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.534273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.534654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.535043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] PATH_INFO: `/v3/projects/44150dcc92c0475ab5f9f4fb1f57c3eb/users/1e78e73e132244ab958d6c54570abc89/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.558981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=44150dcc92c0475ab5f9f4fb1f57c3eb, user_id=1e78e73e132244ab958d6c54570abc89, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.564313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:45.632817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1177/2326] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:45 2022] PUT /identity/v3/projects/44150dcc92c0475ab5f9f4fb1f57c3eb/users/1e78e73e132244ab958d6c54570abc89/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:45.644422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.644876 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.645497 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.686661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] MFA Rules not processed for user `1e78e73e132244ab958d6c54570abc89`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:45.687819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:45.731481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:45.756954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1150/2327] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:42:45.762078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:45.811721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:45.830841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcbec80-068a-4ee1-935f-9e0d676214fc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:45.836043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:45.836687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:45.837013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:45.837315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:45.838289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:45.840398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.034545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1178/2328] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:45 2022] POST /identity/v3/projects => generated 385 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:46.039031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.051345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062b131-4a29-46a8-8e03-89d4ba8a7ddf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.107171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.107828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.108101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.108381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.109054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.111655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.231556 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1151/2329] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:46 2022] POST /identity/v3/users => generated 299 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:46.236311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.246865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2999b-4877-4244-8db9-1ac1e894e543 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.253072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.253738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.254054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.254327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.254977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.256717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.268879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1179/2330] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:46.273037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.285076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4319d9-db14-4104-8b4a-53689d42b269 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.289742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.290392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.290667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.290938 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] PATH_INFO: `/v3/projects/da20b8b86ab44a7e81da13c5368a7665/users/7fc511cc69b54cab8ba71b99ef58bd8b/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.306854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da20b8b86ab44a7e81da13c5368a7665, user_id=7fc511cc69b54cab8ba71b99ef58bd8b, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.309603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.378936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1152/2331] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:46 2022] PUT /identity/v3/projects/da20b8b86ab44a7e81da13c5368a7665/users/7fc511cc69b54cab8ba71b99ef58bd8b/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:46.383017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.393738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4613-d3b7-40dd-befa-61be6dcf4563 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.451285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.451910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.452178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.452452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.453233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.454913 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b5b8c-c30d-4c2e-a38a-1541793cd867 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.465237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1180/2332] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:46.470262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.480760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc596a5-d7c2-4fdb-a645-3c7aa31a4888 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.486202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.486851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.487364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.487506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] PATH_INFO: `/v3/projects/da20b8b86ab44a7e81da13c5368a7665/users/7fc511cc69b54cab8ba71b99ef58bd8b/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.489475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da20b8b86ab44a7e81da13c5368a7665, user_id=7fc511cc69b54cab8ba71b99ef58bd8b, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.492184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.504208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1153/2333] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:46 2022] PUT /identity/v3/projects/da20b8b86ab44a7e81da13c5368a7665/users/7fc511cc69b54cab8ba71b99ef58bd8b/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:46.515389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d113b-f343-458e-9898-d385be311136 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.515713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d113b-f343-458e-9898-d385be311136 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.516024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d113b-f343-458e-9898-d385be311136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.549036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f16d113b-f343-458e-9898-d385be311136 None None] MFA Rules not processed for user `7fc511cc69b54cab8ba71b99ef58bd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:46.550264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f16d113b-f343-458e-9898-d385be311136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:46.624516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1181/2334] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:46.640051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:46.642597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:46.704229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b20552-0b62-492d-83e2-d9c335ef6454 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:46.708035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:46.708650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:46.708929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:46.709204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:46.722115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:46.723927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5fd1c-aea1-4d4e-85c7-0f5467e82ba1 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:46.735055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1154/2335] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:47.827892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d113b-f343-458e-9898-d385be311136 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:47.871922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:47.893171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d113b-f343-458e-9898-d385be311136 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:47.898195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:47.898439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:47.898687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:47.898958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] PATH_INFO: `/v3/users/1e78e73e132244ab958d6c54570abc89` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:47.900124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e78e73e132244ab958d6c54570abc89)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:47.902056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:48.097018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e78e73e132244ab958d6c54570abc89'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.174261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e78e73e132244ab958d6c54570abc89'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.174889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e78e73e132244ab958d6c54570abc89'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.187278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e78e73e132244ab958d6c54570abc89'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.191248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e78e73e132244ab958d6c54570abc89'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.200520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1182/2336] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:47 2022] DELETE /identity/v3/users/1e78e73e132244ab958d6c54570abc89 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:48.205896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:48.206427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:48.206427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:48.226375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:48.227620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:48.262186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:48.289735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1155/2337] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:48.299941 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:48.308277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:48.362410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0775818-5eec-49fc-960e-49d463c04802 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:48.366113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:48.366706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:48.366979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:48.367275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:48.376137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:48.377600 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:48.387121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1183/2338] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:48.826824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:48.828898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:48.842271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5578d64-731e-4312-a157-d3e487c96313 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:48.847913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:48.848312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:48.848812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:48.848970 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] PATH_INFO: `/v3/projects/44150dcc92c0475ab5f9f4fb1f57c3eb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:48.850237 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44150dcc92c0475ab5f9f4fb1f57c3eb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:48.852349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:48.957678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44150dcc92c0475ab5f9f4fb1f57c3eb', 'request_id': 'req-65611634-2b5d-48a7-8f5c-782d28a24126'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:48.964048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44150dcc92c0475ab5f9f4fb1f57c3eb', 'request_id': 'req-65611634-2b5d-48a7-8f5c-782d28a24126'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.013413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44150dcc92c0475ab5f9f4fb1f57c3eb', 'request_id': 'req-65611634-2b5d-48a7-8f5c-782d28a24126'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.024129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] The token cache is being invalidate because project 44150dcc92c0475ab5f9f4fb1f57c3eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:49.024414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.025023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.026506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1156/2339] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:48 2022] DELETE /identity/v3/projects/44150dcc92c0475ab5f9f4fb1f57c3eb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:49.031657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:49.041433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4d21e-1431-45f9-94ab-57bb391ff863 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:49.099517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:49.100977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:49.100977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:49.100977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] PATH_INFO: `/v3/users/7fc511cc69b54cab8ba71b99ef58bd8b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:49.102028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc511cc69b54cab8ba71b99ef58bd8b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:49.104008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:49.105520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:49.280246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc511cc69b54cab8ba71b99ef58bd8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.361875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc511cc69b54cab8ba71b99ef58bd8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.362556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc511cc69b54cab8ba71b99ef58bd8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.376492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc511cc69b54cab8ba71b99ef58bd8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.377129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-eb30d4be-bf36-4a6e-a770-15bd910dce89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc511cc69b54cab8ba71b99ef58bd8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:49.389816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1184/2340] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:42:49 2022] DELETE /identity/v3/users/7fc511cc69b54cab8ba71b99ef58bd8b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:49.790703 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:49.795404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:49.814001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65611634-2b5d-48a7-8f5c-782d28a24126 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:49.874382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:49.875317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:49.875571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:49.875909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] PATH_INFO: `/v3/projects/da20b8b86ab44a7e81da13c5368a7665` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:49.877195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=da20b8b86ab44a7e81da13c5368a7665)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:49.879134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.131175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da20b8b86ab44a7e81da13c5368a7665', 'request_id': 'req-ccdd870d-91f3-4c5c-874a-beffcae8dedf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:50.137564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da20b8b86ab44a7e81da13c5368a7665', 'request_id': 'req-ccdd870d-91f3-4c5c-874a-beffcae8dedf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:50.230271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da20b8b86ab44a7e81da13c5368a7665', 'request_id': 'req-ccdd870d-91f3-4c5c-874a-beffcae8dedf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:50.232574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:50.242422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] The token cache is being invalidate because project da20b8b86ab44a7e81da13c5368a7665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:42:50.242733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:50.243396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ccdd870d-91f3-4c5c-874a-beffcae8dedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:42:50.244919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1157/2341] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:42:49 2022] DELETE /identity/v3/projects/da20b8b86ab44a7e81da13c5368a7665 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:50.251682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.252376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.252376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.273415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:50.274657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.334475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1185/2342] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:50.342682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.343026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.343444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.363425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:50.366904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:50.372732 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1158/2343] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:50.377854 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.387740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6194-9b07-4809-9372-ba5e808c6908 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.392940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.393516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.393798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.394100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.394780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:42:50.395079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.396731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.408420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1186/2344] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:42:50.412605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.422501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3897d-a058-4053-bb72-44c81598de2e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.427307 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.427931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.428206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.428499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.429136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.431081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.569993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1159/2345] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:50 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:50.574288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.584368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75eacf-b51a-4c92-9391-f72cce1e5924 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.640332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.640821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.641101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.641376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.642003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.643954 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.655250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:50.778980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1187/2346] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:50 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:50.783854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.793967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c5262-0451-46ea-bcab-8d113e1bf7d5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.799951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.800582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.800855 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.801121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.801759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.803387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.813998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1160/2347] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:50.818006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.828945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4294d9-6571-4c31-8698-6d66d3cc10a3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.833651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.834256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.834556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.834827 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.836170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.838032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cdb69-8a09-4518-b864-e8368332e96e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.854707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1188/2348] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:50.855949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:50.866968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a18439-c0d2-456d-8178-f644bafa0896 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:50.872103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:50.872596 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.873038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.873148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] PATH_INFO: `/v3/projects/ad92f8a2bbac42b9ae01c0b6935b8682/users/56f7c19fbf7b46e7aabab054a34897dd/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.889230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad92f8a2bbac42b9ae01c0b6935b8682, user_id=56f7c19fbf7b46e7aabab054a34897dd, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:50.892769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:50.956999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1161/2349] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:50 2022] PUT /identity/v3/projects/ad92f8a2bbac42b9ae01c0b6935b8682/users/56f7c19fbf7b46e7aabab054a34897dd/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:50.962571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d27c5-4658-48e7-b898-61d8145f830f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:50.962953 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d27c5-4658-48e7-b898-61d8145f830f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:50.967547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d27c5-4658-48e7-b898-61d8145f830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:50.982264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f30d27c5-4658-48e7-b898-61d8145f830f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:50.983507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f30d27c5-4658-48e7-b898-61d8145f830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:51.042740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1189/2350] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:51.052870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:51.054633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:51.114795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf28548-daad-4f0b-8420-c6d0d676b22e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:51.120018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:51.120654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:51.120928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:51.121209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:51.131610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:51.133048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:51.143300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1162/2351] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:52.510146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:52.510720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:52.510720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:52.541648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] MFA Rules not processed for user `56f7c19fbf7b46e7aabab054a34897dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:52.543001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:52.614034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1190/2352] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:52.675608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:52.685289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aad65-d64e-43a6-9060-88ec1643ff24 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:52.690339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:52.690934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:52.691240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:52.691514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:52.692177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:52.694921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:52.813007 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1163/2353] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:42:52 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:42:52.817533 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:52.829309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ed4af-43cf-434f-a008-b57f188fdf5d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:52.885592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:52.886170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:52.886477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:52.886723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:52.887408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:52.889331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.014465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1191/2354] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:42:52 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:42:53.016461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:53.026749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4ae1b-f159-406b-86a3-973c4bf89b60 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:53.032662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:53.033264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:53.033541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:53.033811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:53.034456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:53.036103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.046206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1164/2355] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:53.050707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:53.057512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:53.061371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f975a2-4b32-4b06-8665-6099704f42fd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:53.068450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:53.069122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:53.069385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:53.069655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] PATH_INFO: `/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68/users/f1dd01dc30f14490b402ba44c4e5ea3e/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:53.088539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5b1f1e1b449ba94091ae67c377b68, user_id=f1dd01dc30f14490b402ba44c4e5ea3e, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:53.091289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.184983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1192/2356] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:53 2022] PUT /identity/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68/users/f1dd01dc30f14490b402ba44c4e5ea3e/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:53.196056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:53.207137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a6ec9-887a-4d89-ae91-5459f6a7a722 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:53.266221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:42:53.278505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:53.279126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:53.279543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:53.279842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:53.280634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:53.282264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394c3fd-2c41-4d0e-ae8e-1678f95dbe41 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.292704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1165/2357] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:42:53.297541 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:53.308210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bfdfa-22a6-45d8-b2c5-6c071aeeffc9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:53.313641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:53.314294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:53.314570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:53.314840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] PATH_INFO: `/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68/users/f1dd01dc30f14490b402ba44c4e5ea3e/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:53.316803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c5b1f1e1b449ba94091ae67c377b68, user_id=f1dd01dc30f14490b402ba44c4e5ea3e, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:53.319483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:53.331802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1193/2358] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:42:53 2022] PUT /identity/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68/users/f1dd01dc30f14490b402ba44c4e5ea3e/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:42:54.788878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.789425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.790764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:54.819905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] MFA Rules not processed for user `f1dd01dc30f14490b402ba44c4e5ea3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:42:54.821172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:54.904352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1166/2359] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:42:54.911548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:54.913671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:54.975366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b04f8-e347-44fe-9c8e-c6c8ba9ff471 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:54.979115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:54.979752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:54.980030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:54.980302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:55.039233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:55.040761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:55.051349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1194/2360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:42:57.842059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:42:57.843579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:42:57.855522 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4cec71-532e-42a9-8989-c21f416b58fa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:42:57.859383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:42:57.859979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:42:57.860259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:42:57.860532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:42:57.869752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:42:57.871199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647aa89-f174-470c-ba7b-73fd6826b174 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:42:57.881745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1167/2361] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:42:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:04.047095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:04.103798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e8b8-42f2-4a4d-a897-793e92fc93c2 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:04.107552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:04.108327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:04.108682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:04.108958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:04.110769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:04.151997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:04.196078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:04.197545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:04.215192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1195/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:06.515118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:06.516016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:06.516184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:06.575661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] MFA Rules not processed for user `8a45ee95cf5947c49584ff403facaff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:06.577551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:06.637391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1168/2363] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:43:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:09.104487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:09.106466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.119044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ad31b-40a3-4787-a7eb-7d5b109ad0c5 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:09.123773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:09.124395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:09.124673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:09.125112 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:09.127085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:09.143524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:09.145321 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:09.158175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1196/2364] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:12.717232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:12.718927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.736061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d495d4-2ea3-44b9-8c12-f0a78d0b2e51 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:12.741816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:12.742426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:12.742722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:12.742994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:12.744759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:12.787729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:12.827601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:12.829314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:12.848799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1169/2365] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:43:18.275933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:18.382088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1f51e-7097-4979-8957-f6cf3faae3df service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:18.390186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:18.391562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:18.392345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:18.393130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] PATH_INFO: `/v3/users/56f7c19fbf7b46e7aabab054a34897dd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:18.394713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f7c19fbf7b46e7aabab054a34897dd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:18.398876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:18.611089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f7c19fbf7b46e7aabab054a34897dd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:18.700441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f7c19fbf7b46e7aabab054a34897dd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:18.701108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f7c19fbf7b46e7aabab054a34897dd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:18.715939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f7c19fbf7b46e7aabab054a34897dd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:18.716880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f7c19fbf7b46e7aabab054a34897dd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:18.729687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1197/2366] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:18 2022] DELETE /identity/v3/users/56f7c19fbf7b46e7aabab054a34897dd => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:43:18.988233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.007763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fea2-9dcf-4738-85ba-310e2fc0f327 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.092744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.093436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.093715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.093987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] PATH_INFO: `/v3/projects/ad92f8a2bbac42b9ae01c0b6935b8682` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.095244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad92f8a2bbac42b9ae01c0b6935b8682)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.097184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.219354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad92f8a2bbac42b9ae01c0b6935b8682', 'request_id': 'req-a159ac3b-3e85-4cb1-9d96-29992a765805'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.225564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad92f8a2bbac42b9ae01c0b6935b8682', 'request_id': 'req-a159ac3b-3e85-4cb1-9d96-29992a765805'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.287999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad92f8a2bbac42b9ae01c0b6935b8682', 'request_id': 'req-a159ac3b-3e85-4cb1-9d96-29992a765805'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.299806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] The token cache is being invalidate because project ad92f8a2bbac42b9ae01c0b6935b8682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:19.300136 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.300968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.302912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1170/2367] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:18 2022] DELETE /identity/v3/projects/ad92f8a2bbac42b9ae01c0b6935b8682 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:19.307136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.309548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:19.324150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd29a0-411d-479c-973f-80e5d0c64e23 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:19.350776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:19.381374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:19.382003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:19.382279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:19.382546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] PATH_INFO: `/v3/users/f1dd01dc30f14490b402ba44c4e5ea3e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:19.383731 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1dd01dc30f14490b402ba44c4e5ea3e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:19.385726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:19.610566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1dd01dc30f14490b402ba44c4e5ea3e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.699564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1dd01dc30f14490b402ba44c4e5ea3e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.700456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1dd01dc30f14490b402ba44c4e5ea3e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.713593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1dd01dc30f14490b402ba44c4e5ea3e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.714427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-67768f7f-dd95-42f9-899f-1818f57f0087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1dd01dc30f14490b402ba44c4e5ea3e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:19.726772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1198/2368] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:43:19 2022] DELETE /identity/v3/users/f1dd01dc30f14490b402ba44c4e5ea3e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:19.987809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:19.990105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:20.010673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a159ac3b-3e85-4cb1-9d96-29992a765805 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.069173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:20.086314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.086955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.087261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.087530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] PATH_INFO: `/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.089248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c5b1f1e1b449ba94091ae67c377b68)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.091606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.225849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c5b1f1e1b449ba94091ae67c377b68', 'request_id': 'req-633bdd66-a8ff-4bd1-826b-65b958528dda'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:20.235659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c5b1f1e1b449ba94091ae67c377b68', 'request_id': 'req-633bdd66-a8ff-4bd1-826b-65b958528dda'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:20.407924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c5b1f1e1b449ba94091ae67c377b68', 'request_id': 'req-633bdd66-a8ff-4bd1-826b-65b958528dda'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:20.419571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] The token cache is being invalidate because project c9c5b1f1e1b449ba94091ae67c377b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:43:20.422268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:20.422268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-633bdd66-a8ff-4bd1-826b-65b958528dda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:43:20.422268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1171/2369] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:43:19 2022] DELETE /identity/v3/projects/c9c5b1f1e1b449ba94091ae67c377b68 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:20.427192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.427739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.427739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.451975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:20.453245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:20.519964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1199/2370] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:20.528578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.528853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.529122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.551195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:20.552415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:20.561363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1172/2371] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:20.565771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.576024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ab5d8-c20a-4028-8f4b-5bfbb4c86a8f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.580803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.581403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.581705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.582238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.584090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:43:20.584090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.584832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.597665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1200/2372] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:43:20.603135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.613517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3309768-e497-4ad5-ac12-5525290df073 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.618298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.619017 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.619425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.620444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.621199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.623431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.720044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1173/2373] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:43:20 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:43:20.724345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.737593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11163a-358d-4b1f-bfa7-f1a4e174bd8f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.783519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:20.809311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.810202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.810590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.810961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.811816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.813998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.911470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1201/2374] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:43:20 2022] POST /identity/v3/users => generated 284 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:43:20.916311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.927615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77e1d7-5883-4dbc-8e49-30f2cd75b67c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.933718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.934332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.934601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.934871 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.935545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.937140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.948799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1174/2375] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:20.953057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:20.965612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0f5b8-23ca-4c76-ab72-9d2d88c68d21 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:20.970893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:20.971524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:20.971797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:20.972065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:20.972712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:20.974348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278b9b3-3bb2-45ab-b55d-4257ce68c59a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:20.986117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1202/2376] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:43:20.990714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:21.002073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db237a-b869-4557-a9a7-93be7207ef31 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:21.006690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:21.007379 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:21.007656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:21.007920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] PATH_INFO: `/v3/projects/b6b3999ac5a241bca37e9330db565731/users/0b3078eea5734732a3141da8f14adb72/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:21.024185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6b3999ac5a241bca37e9330db565731, user_id=0b3078eea5734732a3141da8f14adb72, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:21.028160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:21.111864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1175/2377] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:43:20 2022] PUT /identity/v3/projects/b6b3999ac5a241bca37e9330db565731/users/0b3078eea5734732a3141da8f14adb72/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:43:21.123751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000fb9-8520-4435-b8f7-b02343e1548d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:21.124071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000fb9-8520-4435-b8f7-b02343e1548d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:21.124347 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000fb9-8520-4435-b8f7-b02343e1548d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:21.153707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f3000fb9-8520-4435-b8f7-b02343e1548d None None] MFA Rules not processed for user `0b3078eea5734732a3141da8f14adb72`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:21.154927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3000fb9-8520-4435-b8f7-b02343e1548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:21.224046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1203/2378] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:21.235884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:21.239127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:21.327759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:21.331615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e69e-e683-4c09-b9d8-e917aedcd7c2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:21.337340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:21.338265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:21.338751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:21.339163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:21.351931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:21.354888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:21.369501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1176/2379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:23.375247 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.376132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.376397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.445859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:23.448932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.551385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1204/2380] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:43:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:43:23.776108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:23.779493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:23.946418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594df0ab-291d-4830-ae72-8be5cce7e82d service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:23.952985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:23.953606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:23.953884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:23.954169 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:23.965511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:23.967199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:23.983019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1177/2381] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:43:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:43:33.544671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:33.976049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=45 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:43:41.655848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:41.657304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.730417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be9d9a-aacc-48fb-9c9d-b4a875bdf8bf None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:41.734213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:41.734832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:41.735107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:41.735710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:41.743050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:41.755169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:41.757151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:41.766880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1205/2382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:47.976675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:47.978178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:47.993350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073d0f4-f1e9-405d-855e-078e7ddf8a7b service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:47.999116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e346b0a022ea4218a4bd10c7e6fe6d91', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:47.999841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.000149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.000474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.002377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.084387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.087253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.099465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1178/2383] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:48.315722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.324919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa752669-2089-424e-8e92-fc745994fb10 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.329064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.336388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.336388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.336388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.336388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.342673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.344145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.354270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1206/2384] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:48.430311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.439904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0f71b-60f4-46c2-adbc-bcb416d6193b service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.443754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.444547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.444547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.444834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.446624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.464142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.466658 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.480440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1179/2385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:48.543908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:48.545309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.626137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35864c1d-c334-4371-ab0a-d3956e709725 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:48.631485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:48.632096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:48.632371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:48.632643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:48.634818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:48.647653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:48.649283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21e0dfb-580c-4d72-8162-5dd1e4dd4bee service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:48.659683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1207/2386] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:50.675432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:50.677052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.690954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a842d0-1b5e-43ba-926f-e29e8bdc09b3 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:50.697158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:50.697738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:50.698010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:50.698274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:50.700320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:50.793719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:50.795381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:50.814553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1180/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:43:54.548092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.548680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.548680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.570917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:43:54.572184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:43:54.579766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1208/2388] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:43:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:43:54.592750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:43:54.606079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6e341-5a4e-483a-b979-69bb00938e25 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:43:54.611284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:43:54.611892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:43:54.612165 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:43:54.612457 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:43:54.622297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:43:54.624354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:43:54.633853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1181/2389] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:44:04.564689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=67 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:04.629145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:18.134496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:18.137391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:18.154523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2b4da-d7ae-458e-8a35-93c4864d9eae None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:18.160153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:18.162369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:18.162369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:18.162369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:18.163081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:18.177402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:18.178999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c37ae-d11f-4e7e-b8b3-14c3abc8fbad service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:18.190035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1209/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:18.429534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:18.535792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77024802-8353-4a4e-a8fc-0b0434cb1442 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:18.542220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:18.542872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:18.543166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:18.543456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] PATH_INFO: `/v3/users/0b3078eea5734732a3141da8f14adb72` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:18.544634 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3078eea5734732a3141da8f14adb72)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:18.547486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:18.766506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3078eea5734732a3141da8f14adb72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:18.767610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3078eea5734732a3141da8f14adb72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:18.838006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3078eea5734732a3141da8f14adb72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:18.846174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3078eea5734732a3141da8f14adb72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:18.860640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3078eea5734732a3141da8f14adb72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:18.867051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1182/2391] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:18 2022] DELETE /identity/v3/users/0b3078eea5734732a3141da8f14adb72 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:18.871922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:18.872416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:18.872890 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:18.900613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:18.901815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:18.962612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1210/2392] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:18.973146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:18.974563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:19.032303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89944119-6f21-49a4-80a1-b25280611135 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:19.036015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:19.036680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.036952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.037222 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.046285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:19.047740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0e9a4-5491-4e16-8514-02f32af6a011 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:19.057364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1183/2393] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:19.525928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:19.527500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:19.543391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32780fa2-5a1c-4c28-9a40-3fa8d0404b01 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:19.548117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:19.548755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.549033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.549302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] PATH_INFO: `/v3/projects/b6b3999ac5a241bca37e9330db565731` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.551681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6b3999ac5a241bca37e9330db565731)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:19.553584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:19.650244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6b3999ac5a241bca37e9330db565731', 'request_id': 'req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:19.656433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6b3999ac5a241bca37e9330db565731', 'request_id': 'req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:19.700522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6b3999ac5a241bca37e9330db565731', 'request_id': 'req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:19.711445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] The token cache is being invalidate because project b6b3999ac5a241bca37e9330db565731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:19.712083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:19.712630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-00cac2f0-4a0d-400f-8f59-a7ecc3582cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:19.714134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1211/2394] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:19 2022] DELETE /identity/v3/projects/b6b3999ac5a241bca37e9330db565731 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:19.719107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.719618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.719734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.740548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:19.741971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:19.758913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:19.803890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1184/2395] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:19.810983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.818047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.818047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.832180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:19.833388 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:19.841286 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1212/2396] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:19.845613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:19.855988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24247671-a920-4495-bebb-d1ce67a08154 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:19.860441 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:19.861029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.861301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.861565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.862819 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:19.863431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:19.865005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:19.876975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1185/2397] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:19.882148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:19.891514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e554dd-5192-44a0-94a5-f0ccf8372b1b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:19.897233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:19.897831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:19.898107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:19.898375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:19.899045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:19.900956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.022028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1213/2398] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:19 2022] POST /identity/v3/projects => generated 389 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:20.026236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.036585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01e1a3-ffcd-4492-9553-8ed49437112a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.092889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.093442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.093723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.094001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.094641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.096775 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.239205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1186/2399] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:20 2022] POST /identity/v3/users => generated 295 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:20.243580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.254180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38bd183-10f1-4ffc-a923-5f0aa3bf0e0a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.258746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.259373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.264933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.264933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.264933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.264933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e78b9-89a4-4365-b5fd-3da4ec5371de admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.276024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1214/2400] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:20.277397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.296122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.296640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.296984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.300779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582fb21-02d2-4e19-9f76-6ed29c0622a7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.306332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.306968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.307274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.307539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.308183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.309786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.320598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:20.321761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1187/2401] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:20.321926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.326241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.340664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b523cc2-d5b1-41cc-9009-75b6ac087c3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.345392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.345979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.346256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.346811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] PATH_INFO: `/v3/projects/bab58f751be048fd884afbbf1781af5b/users/f1f51b631e9e44ef874cf2cdef429e9c/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.351588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:20.362675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab58f751be048fd884afbbf1781af5b, user_id=f1f51b631e9e44ef874cf2cdef429e9c, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.365518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f4331-c3bf-48dd-9929-835a81180c96 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.383597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1215/2402] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:20.399026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.399026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.432839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1188/2403] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:20 2022] PUT /identity/v3/projects/bab58f751be048fd884afbbf1781af5b/users/f1f51b631e9e44ef874cf2cdef429e9c/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:20.445963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.446547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.447242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.478261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] MFA Rules not processed for user `f1f51b631e9e44ef874cf2cdef429e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:20.478861 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c5446-e1d4-440d-b075-6f0e3e272e49 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.479536 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.540141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:20.551612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1189/2404] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:20 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:44:20.552601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.553383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.555204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.555204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.562675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:20.564589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:20.646779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.648448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.654524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8d133-d4c5-420e-b557-cadb5bff65fe None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:20.656749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1216/2405] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:44:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 01 16:44:20.658128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:20.659479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:20.659760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:20.660035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:20.669674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:20.671254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a4df6-4216-4be7-afbb-0eee10709150 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:20.682404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1190/2406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:44:30.653348 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:30.677306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=29 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:46.576208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:46.641457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7326d65-3cbe-474f-8c30-3f4207a7ca76 service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:46.646140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:46.646772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.647052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.647359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] PATH_INFO: `/v3/users/f1f51b631e9e44ef874cf2cdef429e9c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.648483 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1f51b631e9e44ef874cf2cdef429e9c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:46.650426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:46.842872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1f51b631e9e44ef874cf2cdef429e9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.914128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1f51b631e9e44ef874cf2cdef429e9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.914846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1f51b631e9e44ef874cf2cdef429e9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.927672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1f51b631e9e44ef874cf2cdef429e9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.929777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1f51b631e9e44ef874cf2cdef429e9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:46.941288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1217/2407] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:44:46 2022] DELETE /identity/v3/users/f1f51b631e9e44ef874cf2cdef429e9c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:46.946289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:46.946850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:46.946962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:46.965986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:46.967309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.030181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1191/2408] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:47.040673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.042094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.103549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1d2cd-a921-44e1-829c-fea04125d2d4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.107826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.108430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.108709 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.108982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.118032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.119550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8d522-38a7-4fe9-9014-8d2ce8b4b926 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.128994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1218/2409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:47.464903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.466405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.480605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24507b-b6d7-42a4-8fa5-605d83eb0f1e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.485859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.487329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.487615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.487886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] PATH_INFO: `/v3/projects/bab58f751be048fd884afbbf1781af5b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.489014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab58f751be048fd884afbbf1781af5b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.491046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.594363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab58f751be048fd884afbbf1781af5b', 'request_id': 'req-18ef0b07-59e9-4d37-99a7-dac709cd502f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.600797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab58f751be048fd884afbbf1781af5b', 'request_id': 'req-18ef0b07-59e9-4d37-99a7-dac709cd502f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.656847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab58f751be048fd884afbbf1781af5b', 'request_id': 'req-18ef0b07-59e9-4d37-99a7-dac709cd502f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.667787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] The token cache is being invalidate because project bab58f751be048fd884afbbf1781af5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:44:47.668070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.668762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18ef0b07-59e9-4d37-99a7-dac709cd502f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:44:47.670318 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1192/2410] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:44:47 2022] DELETE /identity/v3/projects/bab58f751be048fd884afbbf1781af5b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:47.675135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.675642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.675786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.700736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:47.702625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.721389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:47.764639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1219/2411] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:47.773289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.776392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.776392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.793826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:47.795024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:47.802840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1193/2412] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:47.807004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.816741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fa529-27ea-419c-b863-c5e0c212aa70 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.821765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.822360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.822635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.822904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.823583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:44:47.823885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.825485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.837500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1220/2413] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:44:47.841600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.851341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a005ec6-0011-49ed-a554-4aff37fd917e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:47.855907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:47.856495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:47.856774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:47.857038 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:47.857719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:47.859597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:47.945555 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1194/2414] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:47.949922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:47.960767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8d0c6-11dc-4716-be8d-574879591f6a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.017915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.018528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.018807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.019080 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.019756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.022297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.160359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1221/2415] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:47 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:48.165261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.177774 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b27a1f-831a-468d-b235-336e839700ea admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.182625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.183271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.183552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.183825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.184474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.186149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.203676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1195/2416] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:44:48.205098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.217433 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fdbd1-76ba-4bc7-99e4-3deca3417f56 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.222364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.223191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.223364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.223543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.224329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.226229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2fcd6-abe6-4f47-a798-68cf40bc0ef6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.237598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1222/2417] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:48.242143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.253068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159c5c-638c-4a1e-b2a9-a9a3bb2a8674 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.259096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.260045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.260498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.260924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] PATH_INFO: `/v3/projects/a58b7358110c43ebbfbb5b6de842c94b/users/edda03844073424d89deff3eafeae7cb/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.280886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58b7358110c43ebbfbb5b6de842c94b, user_id=edda03844073424d89deff3eafeae7cb, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.283720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.359741 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1196/2418] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:48 2022] PUT /identity/v3/projects/a58b7358110c43ebbfbb5b6de842c94b/users/edda03844073424d89deff3eafeae7cb/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:48.365293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.365579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.365858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.388076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:48.389316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:48.455487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1223/2419] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:48.465502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:48.467912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:48.495228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:48.532697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109bf0d-ffe9-4628-890d-a56b1be25fe4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:48.537336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:48.538289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:48.538624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:48.538895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:48.548302 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:48.549742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f85a9-2109-45c7-a96f-da93c66fd1a0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:48.560816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1197/2420] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:53.052052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:53.060982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.114453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:44:53.176535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57683dec-7395-44d9-9958-421f6b1076ac None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:53.182766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:53.189933 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:53.190312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:53.190683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:53.192940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:53.206260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:53.208207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:53.222512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1224/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:44:54.994529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:54.995143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:54.995143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.029242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] MFA Rules not processed for user `edda03844073424d89deff3eafeae7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:44:55.030655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:44:55.125277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1198/2422] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:44:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:44:55.126663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.139690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59545426-97f3-4d30-949e-d092a7136e80 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.148716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.148716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.149184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.149566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.150326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.152451 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.356264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1225/2423] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:44:55 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:55.360347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.376415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c970882-24b0-4c86-82a2-cd3e31166023 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.460407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.464444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.464444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.464444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.464444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.464444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.546372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1199/2424] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:44:55 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:44:55.548324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.563518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38aa567-f71a-417f-9619-7801ee0f8206 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.570202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.570828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.571110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.571428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.572288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.573975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.585005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1226/2425] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:55.593384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.605006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb847ff-947b-4b6e-a457-c5dfb1a4d792 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.610098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.610702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.610977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.611282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] PATH_INFO: `/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388/users/dfe61da74ec9481683f5aca80ad514a3/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.635072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a3b8290b004e9ab2ecb1bce2b47388, user_id=dfe61da74ec9481683f5aca80ad514a3, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.638225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.728108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1200/2426] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:55 2022] PUT /identity/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388/users/dfe61da74ec9481683f5aca80ad514a3/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:44:55.735194 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.746537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2e761-ec29-4315-b3fd-3299c19f82e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.812705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.813312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.813581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.813860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.814506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.816107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.828740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1227/2427] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:44:55 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:44:55.834351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:55.845145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17ee2c-cc1e-4709-b80b-0e68787be246 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:44:55.851398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:44:55.852068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:44:55.852347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:44:55.852622 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] PATH_INFO: `/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388/users/dfe61da74ec9481683f5aca80ad514a3/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:44:55.854602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a3b8290b004e9ab2ecb1bce2b47388, user_id=dfe61da74ec9481683f5aca80ad514a3, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:44:55.857350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8befb-5547-41a4-a290-9de3cea35c7b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:44:55.873842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1201/2428] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:44:55 2022] PUT /identity/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388/users/dfe61da74ec9481683f5aca80ad514a3/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:44:59.968278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:44:59.970233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:00.046552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972dd35-eff0-4716-9988-7571022ee8ff admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:00.085188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:00.108171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:00.108779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:00.109220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:00.109335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:00.111057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:00.182198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:00.183818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:00.195630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1228/2429] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:44:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:01.208937 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:01.209500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:01.209645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:01.217710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:01.238415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] MFA Rules not processed for user `dfe61da74ec9481683f5aca80ad514a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:01.239684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:01.343739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1202/2430] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:01.359290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:01.485652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c942a45-7a74-404d-9ffa-13ea6e64b6b4 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:01.494409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:01.495662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:01.496169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:01.496658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:01.587068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:01.589016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8be894-2092-4e40-975e-09e1134aba59 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:01.601523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1229/2431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:04.466283 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:04.467809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:04.482804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95c548-0416-41cd-bf99-528ed4ef0355 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:04.490327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:04.490327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:04.490327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:04.490327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] PATH_INFO: `/v3/users/21e6ff7908e84a5994a51be0cfc82414` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:04.490825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e6ff7908e84a5994a51be0cfc82414)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:04.492990 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:04.618284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:04.618799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:04.618916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:04.639008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] MFA Rules not processed for user `edda03844073424d89deff3eafeae7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:04.640267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:04.647806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1230/2432] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:04.710802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e6ff7908e84a5994a51be0cfc82414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:04.710802 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e6ff7908e84a5994a51be0cfc82414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:04.780664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e6ff7908e84a5994a51be0cfc82414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:04.790792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e6ff7908e84a5994a51be0cfc82414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:04.811273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-012c8c25-c0d7-4736-8cb2-1d39fecfe177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e6ff7908e84a5994a51be0cfc82414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:04.822298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1203/2433] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:04 2022] DELETE /identity/v3/users/21e6ff7908e84a5994a51be0cfc82414 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:05.134072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.165374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269b4c1-be5e-4612-8c3b-37bfcc06d277 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.180711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:05.233344 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.233977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.234257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.234528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] PATH_INFO: `/v3/projects/c787c684c86144c792d9afc457ac3212` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.235798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c787c684c86144c792d9afc457ac3212)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.237772 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:05.370326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c787c684c86144c792d9afc457ac3212', 'request_id': 'req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:05.377144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c787c684c86144c792d9afc457ac3212', 'request_id': 'req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:05.449556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c787c684c86144c792d9afc457ac3212', 'request_id': 'req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:05.476028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] The token cache is being invalidate because project c787c684c86144c792d9afc457ac3212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:05.476403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:05.478189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66a3c2ba-e7d1-4761-a6a2-5ded9cb27b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:05.479088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1231/2434] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:05 2022] DELETE /identity/v3/projects/c787c684c86144c792d9afc457ac3212 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:05.485376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.485906 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.486037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.508642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:05.509925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:05.593271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.593570 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.593901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.602072 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1204/2435] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:05.616372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:05.617511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:05.625651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1232/2436] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:05.631363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.636159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:05.641579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c92ff3-4b40-4e05-a08e-8302eea90179 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.646393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.646994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.647530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.647796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.648447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:05.648749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.650399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:05.662172 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1205/2437] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:05.670715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.681171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28445f4-174d-41bf-bc35-a3aadbf5a1ea None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.686289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.686914 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.687229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.687501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.688185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.691133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:05.770391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1233/2438] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:05 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:05.772066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.782760 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b998960-79a3-4996-b212-db301ff72d69 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.847792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.848409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.848686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.848956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.849600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.851575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:05.930516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1206/2439] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:05 2022] POST /identity/v3/users => generated 283 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:05.934537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.946729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926df4a-993c-40f0-b78a-645fb300ff6d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.952034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.952653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.952928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.953199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.953848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.955568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:05.973689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1234/2440] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:05.975091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:05.985794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911b60-e6fd-497e-8b82-c9316625b8d6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:05.992293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:05.992905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:05.993206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:05.993442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:05.994654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:05.996511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf4589f-9eab-401f-ba68-06bcec4452ee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:06.007812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1207/2441] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:06.013907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:06.025319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87f62e-4632-413a-a93f-9eb397282ca3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:06.031237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:06.032129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:06.032402 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:06.032665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] PATH_INFO: `/v3/projects/1c2bf80b9cd74738ac555b55a53066b9/users/e2c4919de643441a9cb6a20eb893dbc2/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:06.052275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2bf80b9cd74738ac555b55a53066b9, user_id=e2c4919de643441a9cb6a20eb893dbc2, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:06.055811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:06.142479 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1235/2442] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:06 2022] PUT /identity/v3/projects/1c2bf80b9cd74738ac555b55a53066b9/users/e2c4919de643441a9cb6a20eb893dbc2/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:06.154932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:06.155419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:06.156304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:06.190414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] MFA Rules not processed for user `e2c4919de643441a9cb6a20eb893dbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:06.191734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:06.285772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1208/2443] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:06.298737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:06.301891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:06.356728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:06.373385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa6348-e608-4279-924b-8a725e699288 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:06.377385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:06.378438 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:06.378856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:06.379140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:06.389845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:06.391335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba632310-89ef-4815-a001-18677396ade8 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:06.402089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1236/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:16.281776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:16.443465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:19.611479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:19.674758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f1b12-9fb1-43f1-af19-bb0291889db8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:19.679335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:19.679971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:19.680248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:19.680525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] PATH_INFO: `/v3/users/e2c4919de643441a9cb6a20eb893dbc2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:19.681642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c4919de643441a9cb6a20eb893dbc2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:19.683618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:19.901213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c4919de643441a9cb6a20eb893dbc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:19.901783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c4919de643441a9cb6a20eb893dbc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:19.973360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c4919de643441a9cb6a20eb893dbc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:19.980910 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c4919de643441a9cb6a20eb893dbc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:19.994375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c4919de643441a9cb6a20eb893dbc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.002189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1209/2445] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:19 2022] DELETE /identity/v3/users/e2c4919de643441a9cb6a20eb893dbc2 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:20.007013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:20.007508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:20.007637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:20.026849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:20.028013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:20.095030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1237/2446] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:20.105258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:20.106891 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:20.177013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd097d1-84ee-47dd-ad28-733d8943a302 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:20.180583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:20.181248 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:20.181529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:20.181796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:20.192965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:20.194490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c5b5a-aa18-476f-a469-31e5c26145a3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:20.205031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1210/2447] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:20.644168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:20.645690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:20.662649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0950e0-3210-434c-abf0-6913219b39b9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:20.673185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:20.673185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:20.673185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:20.673185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] PATH_INFO: `/v3/projects/1c2bf80b9cd74738ac555b55a53066b9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:20.675417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2bf80b9cd74738ac555b55a53066b9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:20.675417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:20.799236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2bf80b9cd74738ac555b55a53066b9', 'request_id': 'req-06b5b39b-634d-4c95-bbc0-bb53fd702190'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.807657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2bf80b9cd74738ac555b55a53066b9', 'request_id': 'req-06b5b39b-634d-4c95-bbc0-bb53fd702190'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.916210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2bf80b9cd74738ac555b55a53066b9', 'request_id': 'req-06b5b39b-634d-4c95-bbc0-bb53fd702190'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.931556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] The token cache is being invalidate because project 1c2bf80b9cd74738ac555b55a53066b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:20.931842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.932535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-06b5b39b-634d-4c95-bbc0-bb53fd702190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:20.934047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1238/2448] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:20 2022] DELETE /identity/v3/projects/1c2bf80b9cd74738ac555b55a53066b9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:20.939478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:20.940041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:20.940041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:20.961470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:20.962580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:20.980942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:21.027297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1211/2449] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:21.037214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.037797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.037797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.061755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:21.063105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:21.071668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1239/2450] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:21.082606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.086280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363db0a1-c394-4781-99cd-024c0caa303f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.090943 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.091567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.091844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.092113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.092752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:21.093049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.095392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.106826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1212/2451] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:21.112124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.121753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf3105-7f69-409b-b9f8-c8ce884fdfb2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.126855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.127501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.137057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.137057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.137057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.137057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.253331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1240/2452] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:21 2022] POST /identity/v3/projects => generated 403 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:21.257557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.273629 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae3701-4b2d-436a-9be0-0a7b7e1cb88f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.345448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.346372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.346823 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.347305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.348367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.352013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.511211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1213/2453] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:21 2022] POST /identity/v3/users => generated 302 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:21.515390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.526425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845eb2c-063f-46b2-b213-d999894c9a66 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.531115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.531829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.532102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.532370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.533011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.534591 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.546200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1241/2454] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:21.551578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.562510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292826-d30a-4088-a761-9a46236a384a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.569494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.570209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.570527 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.570837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.571749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.573908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e23228f-3046-4c0a-9778-6c0de16282a4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.585428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1214/2455] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:21.589957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.600503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f11da-1413-4990-8b20-41945022d38b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.605279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.605964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.606240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.606511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] PATH_INFO: `/v3/projects/55f9dc56825c4f388fce1ecde88996a2/users/1b2f61bf0b184a43962ca3dd97cd474a/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.622663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f9dc56825c4f388fce1ecde88996a2, user_id=1b2f61bf0b184a43962ca3dd97cd474a, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.626134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:21.740288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1242/2456] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:21 2022] PUT /identity/v3/projects/55f9dc56825c4f388fce1ecde88996a2/users/1b2f61bf0b184a43962ca3dd97cd474a/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:21.750984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.751355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.751667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.780997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] MFA Rules not processed for user `1b2f61bf0b184a43962ca3dd97cd474a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:21.783121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:21.848098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1215/2457] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:21.852679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:21.874708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:21.916514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc3cb-483f-4520-a882-69fab2e8b6c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:21.921317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:21.921920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:21.922877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:21.923174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:21.923897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:21.925814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.033340 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1243/2458] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:21 2022] POST /identity/v3/projects => generated 405 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:22.037907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.049220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedbf2db-6a5f-4f27-98f5-f40b214ffe50 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.063306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:22.109678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.110649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.110926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.111225 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.111872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.113813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.231134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1216/2459] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:22 2022] POST /identity/v3/users => generated 309 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:22.235877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.249129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb406193-8f56-436b-90fb-231eb593e083 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.254912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.262305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.262305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.262305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.262305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.262305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.279760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1244/2460] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:22.290436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.295583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37985a2e-f72a-4f51-9a1a-22ab35460354 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.300359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.301013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.301288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.301559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] PATH_INFO: `/v3/projects/43ee9ec098094ff1ac25e910b39f65c2/users/08e7cec03f914ea19a07abb04232eac9/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.317465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ee9ec098094ff1ac25e910b39f65c2, user_id=08e7cec03f914ea19a07abb04232eac9, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.320325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.401630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1217/2461] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:22 2022] PUT /identity/v3/projects/43ee9ec098094ff1ac25e910b39f65c2/users/08e7cec03f914ea19a07abb04232eac9/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:22.405690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.416289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219aebd2-fa6d-4b2a-9856-7a889d8427cd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.488512 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.489121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.489403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.489676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.490317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.491964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f6f1a-d5a8-445c-aac9-d3c8c2cd1281 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.502660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1245/2462] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:22.507201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.518353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f8bc7-4867-4a61-b2f8-9a12bfd35890 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.523123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.523810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.524080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.524347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] PATH_INFO: `/v3/projects/43ee9ec098094ff1ac25e910b39f65c2/users/08e7cec03f914ea19a07abb04232eac9/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.526290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ee9ec098094ff1ac25e910b39f65c2, user_id=08e7cec03f914ea19a07abb04232eac9, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.528982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.540141 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1218/2463] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:22 2022] PUT /identity/v3/projects/43ee9ec098094ff1ac25e910b39f65c2/users/08e7cec03f914ea19a07abb04232eac9/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:22.552762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.552979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.553255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.583175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] MFA Rules not processed for user `08e7cec03f914ea19a07abb04232eac9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:22.584423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:22.628649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.630888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:22.661882 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1246/2464] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:22.675040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:22.677068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:22.697809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:22.716041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95c9-93ac-40b3-8a0c-9fc5f0e32b1e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.722693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.723355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.723624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.723902 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.725637 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:22.739138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4225b-6631-46f9-887c-1b075780f655 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:22.742887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:22.743521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:22.743884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:22.744071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:22.754272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.755740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.767713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1247/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:22 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:22.795428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:22.798736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5409f6b-7a24-47b0-af56-0c152b106ff8 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:22.819403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:22.825656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1219/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:22.941569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:23.008409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014626f1-d7ee-4e31-9c6d-93580ff3bcf4 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:23.013331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:23.013931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:23.014206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:23.014474 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] PATH_INFO: `/v3/users/1b2f61bf0b184a43962ca3dd97cd474a` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:23.015617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2f61bf0b184a43962ca3dd97cd474a)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:23.018215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:23.210917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2f61bf0b184a43962ca3dd97cd474a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.253419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2f61bf0b184a43962ca3dd97cd474a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.254682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2f61bf0b184a43962ca3dd97cd474a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.270641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2f61bf0b184a43962ca3dd97cd474a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.271406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2f61bf0b184a43962ca3dd97cd474a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:23.284232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1248/2467] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:22 2022] DELETE /identity/v3/users/1b2f61bf0b184a43962ca3dd97cd474a => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:23.288916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:23.289456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:23.289456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:23.309964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:23.311192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:23.394184 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1220/2468] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:23.403864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:23.405283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:23.454301 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:23.473850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee665cc-62b4-4413-9980-b26387ccfe5b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:23.477946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:23.478612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:23.478825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:23.479349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:23.490352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:23.495506 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:23.502633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1249/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:23.929986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:23.934432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:23.947590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f20a96-9b94-496a-94d2-5044a6bdd6e7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:23.955649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:23.956296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:23.956565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:23.956835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] PATH_INFO: `/v3/projects/55f9dc56825c4f388fce1ecde88996a2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:23.957958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f9dc56825c4f388fce1ecde88996a2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:23.960788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:24.173904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f9dc56825c4f388fce1ecde88996a2', 'request_id': 'req-2827c400-df75-447b-8057-d7ec1812e1b5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.180070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f9dc56825c4f388fce1ecde88996a2', 'request_id': 'req-2827c400-df75-447b-8057-d7ec1812e1b5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.241359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f9dc56825c4f388fce1ecde88996a2', 'request_id': 'req-2827c400-df75-447b-8057-d7ec1812e1b5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.256518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] The token cache is being invalidate because project 55f9dc56825c4f388fce1ecde88996a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:24.258434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.258434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.258993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1221/2470] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:23 2022] DELETE /identity/v3/projects/55f9dc56825c4f388fce1ecde88996a2 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:24.263670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:24.274626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d702f-be6c-471b-a89a-59e1e7b19245 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:24.334173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:24.334799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:24.335073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:24.335372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] PATH_INFO: `/v3/users/08e7cec03f914ea19a07abb04232eac9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:24.336480 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e7cec03f914ea19a07abb04232eac9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:24.338484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:24.490168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e7cec03f914ea19a07abb04232eac9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.565868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e7cec03f914ea19a07abb04232eac9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.566790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e7cec03f914ea19a07abb04232eac9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.582219 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e7cec03f914ea19a07abb04232eac9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.583076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f521facb-c9bb-41e0-9c1c-838047f29662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e7cec03f914ea19a07abb04232eac9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:24.595886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1250/2471] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:24 2022] DELETE /identity/v3/users/08e7cec03f914ea19a07abb04232eac9 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:25.027053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.028753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.042931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827c400-df75-447b-8057-d7ec1812e1b5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.108179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.108821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.109098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.109369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] PATH_INFO: `/v3/projects/43ee9ec098094ff1ac25e910b39f65c2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.110603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ee9ec098094ff1ac25e910b39f65c2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.112564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.137238 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:25.266773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ee9ec098094ff1ac25e910b39f65c2', 'request_id': 'req-99e2bc5f-0058-45d7-8033-85e4de582d72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.273960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ee9ec098094ff1ac25e910b39f65c2', 'request_id': 'req-99e2bc5f-0058-45d7-8033-85e4de582d72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.355597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ee9ec098094ff1ac25e910b39f65c2', 'request_id': 'req-99e2bc5f-0058-45d7-8033-85e4de582d72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.367616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] The token cache is being invalidate because project 43ee9ec098094ff1ac25e910b39f65c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:25.367903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.368547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99e2bc5f-0058-45d7-8033-85e4de582d72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:25.370057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1222/2472] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:25 2022] DELETE /identity/v3/projects/43ee9ec098094ff1ac25e910b39f65c2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:25.375875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.376740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.376740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.397144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:25.398368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.452152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:25.461637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1251/2473] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:25.471049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.473916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.473916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.494804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:25.496843 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:25.514766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1223/2474] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:25.516682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.528830 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b78725-6b02-4c85-9338-bd44dba0df36 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.533948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.534556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.534833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.535129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.535820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:25.536119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.537814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.551396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1252/2475] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:25.556424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.570099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6fa88-a124-4c5c-962b-6351f9041e08 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.577545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.579102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.579557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.579967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.581734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.584946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.712270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1224/2476] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:25.717121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.728105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da049d1d-4ec1-4154-9e91-799663f11771 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.788386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.788741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.789014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.789284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.789926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.792060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.917134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1253/2477] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:25 2022] POST /identity/v3/users => generated 303 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:25.922147 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.933330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7077c-5cb0-48b0-b151-b150b7d82821 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.939016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.939694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.939969 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.940240 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.940888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.942872 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.953575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1225/2478] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:25.957994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:25.968751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397be31-322d-4ba1-a21b-b30f41f8ebda admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:25.976227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:25.976227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:25.976227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:25.976227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:25.976641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:25.977475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883f81-e28e-4b96-b74e-2bfd18d411ae admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:25.989952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1254/2479] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:25.994211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.005059 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165857fe-d1b4-4587-90fb-5efcf7b3938b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.009598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.011415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.011415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.011415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] PATH_INFO: `/v3/projects/a189401ce5df4a6681fa512d818063ce/users/231556dee9ee45b7ae539c9d8e42c792/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.025560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a189401ce5df4a6681fa512d818063ce, user_id=231556dee9ee45b7ae539c9d8e42c792, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.028266 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.161365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1226/2480] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:25 2022] PUT /identity/v3/projects/a189401ce5df4a6681fa512d818063ce/users/231556dee9ee45b7ae539c9d8e42c792/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:26.167004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acf27-f2f1-4180-9276-f1763b8eb527 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.167370 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acf27-f2f1-4180-9276-f1763b8eb527 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.167762 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186acf27-f2f1-4180-9276-f1763b8eb527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.187416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-186acf27-f2f1-4180-9276-f1763b8eb527 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:26.188918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-186acf27-f2f1-4180-9276-f1763b8eb527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:26.254649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1255/2481] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:26.264666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:26.267393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:26.335386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27d40c-3c98-44df-9fea-0fc35691d035 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:26.339468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:26.340085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:26.340365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:26.340641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:26.351439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:26.352889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:26.364265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1227/2482] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:27.786056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:27.786577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:27.786577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:27.818793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] MFA Rules not processed for user `231556dee9ee45b7ae539c9d8e42c792`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:27.820951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:27.879176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:27.889915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1256/2483] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:27 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:27.894442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:27.906355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718ebb5-7294-4fa3-b249-4d26844c1068 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:27.911228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:27.911838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:27.912115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:27.912387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:27.913041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:27.914952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:27.997395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1228/2484] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:27 2022] POST /identity/v3/projects => generated 403 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:28.001810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:28.012918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7303f16c-59ed-416f-ab05-489f46c0b4e5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:28.081081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:28.081970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:28.081970 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:28.082196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:28.082945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:28.084936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:28.177934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1257/2485] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:28 2022] POST /identity/v3/users => generated 308 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:28.182090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:28.193109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded8899b-a27d-4e17-86c9-ffd4834e6a51 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:28.197751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:28.198368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:28.198647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:28.198919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:28.199610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:28.201209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:28.212684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1229/2486] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:28.217090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:28.228043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46719ac0-3b1b-4712-80c4-048c471c7ec7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:28.232717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:28.233367 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:28.233650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:28.234427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] PATH_INFO: `/v3/projects/25291ea6b25648bfaca56c42b61a5b7b/users/70a1577713334b88ab5ad2cc5efe994b/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:28.250987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25291ea6b25648bfaca56c42b61a5b7b, user_id=70a1577713334b88ab5ad2cc5efe994b, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:28.253716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:28.333048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1258/2487] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:28 2022] PUT /identity/v3/projects/25291ea6b25648bfaca56c42b61a5b7b/users/70a1577713334b88ab5ad2cc5efe994b/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:28.337303 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:28.347767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db836d0-922c-4d60-975d-78efd568ca34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:28.362352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:28.410256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:28.410920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:28.411226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:28.411499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:28.412137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:28.413746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762b693-920d-4dfd-9a2f-9c3d324f3ce5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:28.425296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1230/2488] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:28.429307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:28.443908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956eedf-10c8-4945-b74d-36b4b7c28240 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:28.449811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:28.450473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:28.450756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:28.451018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] PATH_INFO: `/v3/projects/25291ea6b25648bfaca56c42b61a5b7b/users/70a1577713334b88ab5ad2cc5efe994b/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:28.454056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25291ea6b25648bfaca56c42b61a5b7b, user_id=70a1577713334b88ab5ad2cc5efe994b, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:28.459607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:28.471792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1259/2489] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:28 2022] PUT /identity/v3/projects/25291ea6b25648bfaca56c42b61a5b7b/users/70a1577713334b88ab5ad2cc5efe994b/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:29.852448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:29.853290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:29.853488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:29.895385 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] MFA Rules not processed for user `70a1577713334b88ab5ad2cc5efe994b`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:29.896860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:29.993174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1231/2490] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:29 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:30.558989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:30.561049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.643787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dbe55-5019-463d-9554-b37310bffb49 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:30.649609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:30.650213 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:30.650488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:30.650752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:30.652503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:30.673687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:30.673687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:30.682858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1260/2491] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:31.476714 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:31.538353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67038e3-a1ce-421d-9d0e-a31599693ddf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:31.542284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:31.542915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:31.543250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:31.543552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:31.546958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:31.553847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:31.557020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:31.558574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:31.570667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1232/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:31.600711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:31.623549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83896c44-e3fb-47c1-b1b4-591d118e7d47 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:31.629386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:31.630045 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:31.630893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:31.631177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] PATH_INFO: `/v3/users/231556dee9ee45b7ae539c9d8e42c792` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:31.632371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=231556dee9ee45b7ae539c9d8e42c792)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:31.634468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:31.800710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '231556dee9ee45b7ae539c9d8e42c792'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:31.875684 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '231556dee9ee45b7ae539c9d8e42c792'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:31.876319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '231556dee9ee45b7ae539c9d8e42c792'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:31.900977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '231556dee9ee45b7ae539c9d8e42c792'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:31.901907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '231556dee9ee45b7ae539c9d8e42c792'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:31.919006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1261/2493] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:31 2022] DELETE /identity/v3/users/231556dee9ee45b7ae539c9d8e42c792 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:32.255621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:32.285783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13884c7-40d8-4530-8eed-284127c198c7 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:32.350444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:32.369104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:32.370040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:32.370384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:32.370769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] PATH_INFO: `/v3/projects/a189401ce5df4a6681fa512d818063ce` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:32.372221 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a189401ce5df4a6681fa512d818063ce)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:32.374824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:32.507689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a189401ce5df4a6681fa512d818063ce', 'request_id': 'req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:32.524685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a189401ce5df4a6681fa512d818063ce', 'request_id': 'req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:32.622972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a189401ce5df4a6681fa512d818063ce', 'request_id': 'req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:32.636304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] The token cache is being invalidate because project a189401ce5df4a6681fa512d818063ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:32.636586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:32.637280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:32.638889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1233/2494] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:32 2022] DELETE /identity/v3/projects/a189401ce5df4a6681fa512d818063ce => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:32.643259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:32.644834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:32.658310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb67733-cb19-4ce6-a4ca-c4846b1afe72 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:32.720118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:32.720857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:32.721183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:32.721500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] PATH_INFO: `/v3/users/70a1577713334b88ab5ad2cc5efe994b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:32.722855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a1577713334b88ab5ad2cc5efe994b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:32.724985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:32.931107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a1577713334b88ab5ad2cc5efe994b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.000403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a1577713334b88ab5ad2cc5efe994b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.003817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a1577713334b88ab5ad2cc5efe994b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.006192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:33.015127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a1577713334b88ab5ad2cc5efe994b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.016187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-93e3c548-5595-413a-9589-5ff16661362e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a1577713334b88ab5ad2cc5efe994b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.029212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1262/2495] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:32 2022] DELETE /identity/v3/users/70a1577713334b88ab5ad2cc5efe994b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:33.333514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:33.335371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:33.361074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19fc2b-7b56-44e4-a3c2-3dcc90af0a1f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:33.438380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:33.438992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:33.439299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:33.439573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] PATH_INFO: `/v3/projects/25291ea6b25648bfaca56c42b61a5b7b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:33.440704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25291ea6b25648bfaca56c42b61a5b7b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:33.442933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:33.559794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25291ea6b25648bfaca56c42b61a5b7b', 'request_id': 'req-3939bc8e-f5ac-425e-a743-a8880341c5a9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.582101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25291ea6b25648bfaca56c42b61a5b7b', 'request_id': 'req-3939bc8e-f5ac-425e-a743-a8880341c5a9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.650626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25291ea6b25648bfaca56c42b61a5b7b', 'request_id': 'req-3939bc8e-f5ac-425e-a743-a8880341c5a9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.662510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] The token cache is being invalidate because project 25291ea6b25648bfaca56c42b61a5b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:33.662799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.663470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3939bc8e-f5ac-425e-a743-a8880341c5a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:33.665139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1234/2496] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:33 2022] DELETE /identity/v3/projects/25291ea6b25648bfaca56c42b61a5b7b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:33.670138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:33.670660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:33.670759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:33.691701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:33.692895 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:33.758170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1263/2497] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:33.767180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:33.767526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:33.767849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:33.775682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:33.788146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:33.789329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:33.797359 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1235/2498] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:33.802368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:33.814724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0753c08-98fa-4f5f-abb3-3d4827d04b70 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:33.819859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:33.820468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:33.820749 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:33.821019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:33.821679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:33.821978 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:33.823630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:33.836858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1264/2499] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:33.841923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:33.852009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027aa81-315c-4f9b-aed0-d3c78a86ddd2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:33.856949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:33.860132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:33.860132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:33.860132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:33.860132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:33.860824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:33.986752 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1236/2500] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:33 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:33.988465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.005733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9592e-80b2-4031-830b-501647ee5321 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.070078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.070751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.071122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.071631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.072240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.074592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:34.182847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1265/2501] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:33 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:34.184907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.204335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab498b9a-a6d4-4bc8-b3d7-e33f9cf3c987 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.211614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.212328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.212710 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.213116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.213865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.217407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:34.236314 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1237/2502] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:34.240166 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.262278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f7694-541d-4156-919b-181303574874 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.271718 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.272810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.283339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.283339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.283339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.283339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85294ac8-4c7d-48a4-bb6e-1f3794433966 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:34.287580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1266/2503] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:34.292352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.310090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b16dd-da67-4ef1-a78f-33b676c1006e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.318406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.319470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.319934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.320804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] PATH_INFO: `/v3/projects/b1b5bb7d81f04dbabbacf09253c9d1ac/users/0efaf5f9ed43477aab15da1e012b21a8/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.350477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b5bb7d81f04dbabbacf09253c9d1ac, user_id=0efaf5f9ed43477aab15da1e012b21a8, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.355627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:34.511035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1238/2504] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:34 2022] PUT /identity/v3/projects/b1b5bb7d81f04dbabbacf09253c9d1ac/users/0efaf5f9ed43477aab15da1e012b21a8/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:34.521407 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.521692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.521967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.557381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] MFA Rules not processed for user `0efaf5f9ed43477aab15da1e012b21a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:34.558662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:34.595516 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:34.633498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1267/2505] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:34.638320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.708921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9ea54-da18-4e15-b76c-7a9c0bad722f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.716856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.717825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.718041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.718218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.719048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.721043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:34.871699 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1239/2506] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:34 2022] POST /identity/v3/projects => generated 371 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:34.876352 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:34.887432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be45dc8c-44f2-4490-91fb-bc31ff971f8c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:34.957375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:34.958003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:34.958281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:34.958594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:34.959266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:34.961329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.111007 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1268/2507] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:34 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:35.115023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.126449 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35242290-ce09-45dd-8e74-58cbaa97986c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.131090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.132104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.132104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.132322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.132883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:35.134479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.148587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1240/2508] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:35.152849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.164279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1f19-3759-4d4e-a27e-df58ebd67e62 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.168980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.169639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.169916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.170193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] PATH_INFO: `/v3/projects/525d255782d745bcacc3a1f046e84d3f/users/72c64b0fb85a461986f29a888bd6816f/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.186892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525d255782d745bcacc3a1f046e84d3f, user_id=72c64b0fb85a461986f29a888bd6816f, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:35.189701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.303857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1269/2509] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:35 2022] PUT /identity/v3/projects/525d255782d745bcacc3a1f046e84d3f/users/72c64b0fb85a461986f29a888bd6816f/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:35.307980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.318613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a8a37-b6eb-4eb3-9766-386d70a5c33b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.377573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.378280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.378631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.379019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.379771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:35.381477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c73cac-5e6f-4d68-9a32-084d63d1520f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.395669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1241/2510] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:45:35.396872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.408175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00c52f-8287-4d46-a125-b5185dba4e14 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.413011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.413664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.413945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.414796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] PATH_INFO: `/v3/projects/525d255782d745bcacc3a1f046e84d3f/users/72c64b0fb85a461986f29a888bd6816f/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.416905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=525d255782d745bcacc3a1f046e84d3f, user_id=72c64b0fb85a461986f29a888bd6816f, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:35.419596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.431212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1270/2511] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:35 2022] PUT /identity/v3/projects/525d255782d745bcacc3a1f046e84d3f/users/72c64b0fb85a461986f29a888bd6816f/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:35.451822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.452156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.452351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.484904 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:35.493641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] MFA Rules not processed for user `72c64b0fb85a461986f29a888bd6816f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:35.494862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:35.573568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1242/2512] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:35.583417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:35.585132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:35.648119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b25a3-c687-4460-b09f-ac2d6a43f33e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:35.651867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:35.652477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:35.652775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:35.653047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:35.663765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:35.665203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3e458-8525-441b-89b2-0030d43a4979 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:35.675860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1271/2513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:36.731860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:36.802607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbcb54-fdff-4055-927e-a8d0e0d1377b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:36.807433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:36.808893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:36.808893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:36.808893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] PATH_INFO: `/v3/users/0efaf5f9ed43477aab15da1e012b21a8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:36.809842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0efaf5f9ed43477aab15da1e012b21a8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:36.811886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:36.996744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0efaf5f9ed43477aab15da1e012b21a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:36.997434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0efaf5f9ed43477aab15da1e012b21a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:37.069419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0efaf5f9ed43477aab15da1e012b21a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:37.083200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0efaf5f9ed43477aab15da1e012b21a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:37.111176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0efaf5f9ed43477aab15da1e012b21a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:37.121877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1243/2514] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:36 2022] DELETE /identity/v3/users/0efaf5f9ed43477aab15da1e012b21a8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:37.127909 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:37.129456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:37.130188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:37.159144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:37.161676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:37.208588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:37.278494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1272/2515] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:37.290643 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:37.292856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:37.310382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:37.372580 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c2430-4451-4406-a4ae-347251c3c83f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:37.377552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:37.378368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:37.378873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:37.379309 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:37.391310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:37.393063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:37.404853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1244/2516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:38.058889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:38.060129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:38.075805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e642b-4f3a-45cd-a84c-7fb3c1f6bff3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:38.117879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:38.118649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:38.118930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:38.119231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] PATH_INFO: `/v3/projects/b1b5bb7d81f04dbabbacf09253c9d1ac` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:38.120373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b5bb7d81f04dbabbacf09253c9d1ac)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:38.122404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:38.351163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b5bb7d81f04dbabbacf09253c9d1ac', 'request_id': 'req-19951efc-b306-41b6-9182-72055d5289e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:38.360874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b5bb7d81f04dbabbacf09253c9d1ac', 'request_id': 'req-19951efc-b306-41b6-9182-72055d5289e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:38.453536 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b5bb7d81f04dbabbacf09253c9d1ac', 'request_id': 'req-19951efc-b306-41b6-9182-72055d5289e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:38.480151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] The token cache is being invalidate because project b1b5bb7d81f04dbabbacf09253c9d1ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:38.480546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:38.481327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:38.484035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1273/2517] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:38 2022] DELETE /identity/v3/projects/b1b5bb7d81f04dbabbacf09253c9d1ac => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:38.488148 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:38.516476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dcabb-6715-479e-b148-983ce5c49688 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:38.519123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:38.521167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:38.699528 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19951efc-b306-41b6-9182-72055d5289e3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:38.707860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:38.708498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:38.708776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:38.709051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:38.710863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:38.712544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:38.713820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:38.714094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:38.714372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] PATH_INFO: `/v3/users/72c64b0fb85a461986f29a888bd6816f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:38.720081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c64b0fb85a461986f29a888bd6816f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:38.725473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:38.727889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:38.730125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:38.763223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1274/2518] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:39.039173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c64b0fb85a461986f29a888bd6816f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:39.039879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c64b0fb85a461986f29a888bd6816f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:39.166697 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c64b0fb85a461986f29a888bd6816f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:39.177119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c64b0fb85a461986f29a888bd6816f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:39.196338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2175824e-3d40-489c-b683-f0ac1a539863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c64b0fb85a461986f29a888bd6816f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:39.205567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1245/2519] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:38 2022] DELETE /identity/v3/users/72c64b0fb85a461986f29a888bd6816f => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:39.877811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:39.879585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:39.893508 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604589-23d8-4466-8d74-7054f0411e8f service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:39.939728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:39.970302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:39.971266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:39.971635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:39.972023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] PATH_INFO: `/v3/projects/525d255782d745bcacc3a1f046e84d3f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:39.973346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] RBAC: Authorizing `identity:delete_project(project_id=525d255782d745bcacc3a1f046e84d3f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:39.976079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.146880 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525d255782d745bcacc3a1f046e84d3f', 'request_id': 'req-b892f383-094a-46d6-bfa2-2574edb5370e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:40.153465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525d255782d745bcacc3a1f046e84d3f', 'request_id': 'req-b892f383-094a-46d6-bfa2-2574edb5370e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:40.205597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525d255782d745bcacc3a1f046e84d3f', 'request_id': 'req-b892f383-094a-46d6-bfa2-2574edb5370e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:40.219158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] The token cache is being invalidate because project 525d255782d745bcacc3a1f046e84d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:40.219470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:40.220130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b892f383-094a-46d6-bfa2-2574edb5370e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:40.221815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1275/2520] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:39 2022] DELETE /identity/v3/projects/525d255782d745bcacc3a1f046e84d3f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:40.226654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.227170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.227373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.267825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:40.269330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:40.313142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:40.355357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1246/2521] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:40.363884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.378951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.378951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.414752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:40.414752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:40.423414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1276/2522] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:40.427912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.438080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78b63d-622e-46f3-a6c8-b59df3b54d1e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.443961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.444664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.444998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.445330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.446205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:40.446574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.448279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.459936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1247/2523] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:40.465711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.476759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783b791-3874-4bce-807f-e7eccda74325 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.481672 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.482283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.482800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.483078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.484517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.486667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.587339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1277/2524] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:40 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:40.592730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.606138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca584e-54de-4afd-bbdf-517deb7713a3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.704991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.705716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.706005 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.706276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.707132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.709845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.814644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1248/2525] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:40 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:40.816289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.826630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c277d-2d4f-46b9-91b7-03c1109fc00c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.832139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.832734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.833009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.833280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.833920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.835554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.847750 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1278/2526] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:40.858001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.870836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc524-b4dc-4c7e-adc2-e425042e37a3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.876951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.877583 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.877870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.878143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.878800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.880496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d81d27-565a-4970-b367-0fd7873b6922 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:40.893883 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1249/2527] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:40.898096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:40.909208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b5778-4ec9-4077-9e91-4378233df791 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:40.914178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:40.914845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:40.915130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:40.915439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] PATH_INFO: `/v3/projects/4ab50a79896341ae82765120d1778513/users/81936117f5164e48b8ef5900aeca85c7/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:40.932496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab50a79896341ae82765120d1778513, user_id=81936117f5164e48b8ef5900aeca85c7, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:40.935179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.039746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1279/2528] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:40 2022] PUT /identity/v3/projects/4ab50a79896341ae82765120d1778513/users/81936117f5164e48b8ef5900aeca85c7/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:41.048236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.048236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.048521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.083716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] MFA Rules not processed for user `81936117f5164e48b8ef5900aeca85c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:41.085121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:41.168369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1250/2529] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:41.173291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.273476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fcbdc-5793-46b2-84e7-01aa203b8cb6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.278931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.279942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.280173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.280450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.281250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.283561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.292484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:41.460908 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1280/2530] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:41 2022] POST /identity/v3/projects => generated 385 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:41.465601 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.481416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b6ac-ab5d-4565-81ed-382c2d510873 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.524347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:41.560350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.561089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.561440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.561781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.563349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.565909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.653664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.667833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1251/2531] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:41 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:41.673812 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3f5e6-49f8-4114-8862-2e8f5ec749e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.679999 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.680616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.680892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.681161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.681808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.683666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.700041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1281/2532] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:41.700567 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.713884 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51692599-22fe-4f55-94f3-addb71700682 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.718920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.720023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.720517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.720973 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] PATH_INFO: `/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b/users/2e26a99e47734b5b915278c0e9eea443/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.740233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba5c6e2f0bb44c69946a845ac985a9b, user_id=2e26a99e47734b5b915278c0e9eea443, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.743475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.806035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1252/2533] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:41 2022] PUT /identity/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b/users/2e26a99e47734b5b915278c0e9eea443/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:41.809922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.825903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d209c2-affa-4cd3-acdc-b55e91c6bfcf admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.895332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.896322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.896778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.897668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.897887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.899543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5f86f-0a32-402f-a930-1499fa4a3159 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.922679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1282/2534] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:41 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:41.939627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:41.949918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16bacd-ef5d-439c-b018-953aab7fbb85 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:41.957818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:41.957818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.957818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.957818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] PATH_INFO: `/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b/users/2e26a99e47734b5b915278c0e9eea443/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:41.959106 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba5c6e2f0bb44c69946a845ac985a9b, user_id=2e26a99e47734b5b915278c0e9eea443, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:41.961974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:41.974488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1253/2535] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:41 2022] PUT /identity/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b/users/2e26a99e47734b5b915278c0e9eea443/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:41.986444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:41.988907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:41.989241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:42.024420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] MFA Rules not processed for user `2e26a99e47734b5b915278c0e9eea443`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:42.025773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:42.108503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1283/2536] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:42.121589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:42.123525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:42.200492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444fb9b-7940-4a34-bf8e-79ead498abcd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:42.210025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:42.210685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:42.210965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:42.211589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:42.224791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:42.226331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862826f-b96c-41b3-b452-6930c0d06f45 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:42.240270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1254/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:42.330667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:42.392171 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:42.396569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f676a-d162-4f98-90d4-f13af1f2905c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:42.401092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:42.401806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:42.402082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:42.402353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] PATH_INFO: `/v3/users/81936117f5164e48b8ef5900aeca85c7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:42.403554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=81936117f5164e48b8ef5900aeca85c7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:42.405652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:42.693549 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81936117f5164e48b8ef5900aeca85c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:42.838278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81936117f5164e48b8ef5900aeca85c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:42.838996 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81936117f5164e48b8ef5900aeca85c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:42.852332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81936117f5164e48b8ef5900aeca85c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:42.853044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81936117f5164e48b8ef5900aeca85c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:42.866252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1284/2538] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:42 2022] DELETE /identity/v3/users/81936117f5164e48b8ef5900aeca85c7 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:42.880770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:42.881210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:42.881368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:42.903014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:42.904278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:42.976867 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1255/2539] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:42.982666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:42.985055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:43.055250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae7b00-d9e7-42f3-9bb4-4b0feb6350fa admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:43.059659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:43.060269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:43.060596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:43.060896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:43.074144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:43.075725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:43.087274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1285/2540] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:43.645835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:43.647468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:43.662192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c36dc-9090-46cd-bcbb-074b4df8c8f4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:43.667936 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:43.668582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:43.668859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:43.669128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] PATH_INFO: `/v3/projects/4ab50a79896341ae82765120d1778513` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:43.670254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab50a79896341ae82765120d1778513)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:43.672945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:43.701456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:43.790763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab50a79896341ae82765120d1778513', 'request_id': 'req-099e096c-ef2a-4162-b499-feff67b2d66b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:43.797877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab50a79896341ae82765120d1778513', 'request_id': 'req-099e096c-ef2a-4162-b499-feff67b2d66b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:43.891862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab50a79896341ae82765120d1778513', 'request_id': 'req-099e096c-ef2a-4162-b499-feff67b2d66b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:43.903258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] The token cache is being invalidate because project 4ab50a79896341ae82765120d1778513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:43.903552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:43.904213 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:43.906477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1256/2541] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:43 2022] DELETE /identity/v3/projects/4ab50a79896341ae82765120d1778513 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:43.910592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:43.920774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafffc6-8cc7-488b-9ccd-7d3427f4b9d4 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:43.979309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:43.979975 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:43.980255 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:43.980526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] PATH_INFO: `/v3/users/2e26a99e47734b5b915278c0e9eea443` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:43.981738 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e26a99e47734b5b915278c0e9eea443)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:43.983769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:44.008834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:44.234204 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e26a99e47734b5b915278c0e9eea443'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:44.336409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e26a99e47734b5b915278c0e9eea443'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:44.337230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e26a99e47734b5b915278c0e9eea443'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:44.351806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e26a99e47734b5b915278c0e9eea443'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:44.352463 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fada8f21-bc6e-4bcf-8488-15de47bafa9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e26a99e47734b5b915278c0e9eea443'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:44.364981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1286/2542] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:43 2022] DELETE /identity/v3/users/2e26a99e47734b5b915278c0e9eea443 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:45.037585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:45.040880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:45.058158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e096c-ef2a-4162-b499-feff67b2d66b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:45.132997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:45.133654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.133932 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.134210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] PATH_INFO: `/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.135507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba5c6e2f0bb44c69946a845ac985a9b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:45.137628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:45.271977 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba5c6e2f0bb44c69946a845ac985a9b', 'request_id': 'req-41444cd3-1352-4380-8f49-1debdd0db770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:45.279464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba5c6e2f0bb44c69946a845ac985a9b', 'request_id': 'req-41444cd3-1352-4380-8f49-1debdd0db770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:45.427933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba5c6e2f0bb44c69946a845ac985a9b', 'request_id': 'req-41444cd3-1352-4380-8f49-1debdd0db770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:45.440796 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] The token cache is being invalidate because project cba5c6e2f0bb44c69946a845ac985a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:45.441262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:45.442132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41444cd3-1352-4380-8f49-1debdd0db770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:45.444020 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1257/2543] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:45 2022] DELETE /identity/v3/projects/cba5c6e2f0bb44c69946a845ac985a9b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:45.449694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.450151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.450562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.486927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:45.486927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:45.556688 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1287/2544] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:45.573001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.573286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.573564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.602032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:45.603358 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:45.613933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1258/2545] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:45.619983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:45.634613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19aff5-7e0e-411b-a157-632eee0606fa None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:45.639566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:45.641016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.641582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.641939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.642604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:45.642927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:45.644550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:45.659619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1288/2546] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:45.664276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:45.675494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafade98-6275-403f-83a6-707e194ee3e5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:45.680675 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:45.681280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.681554 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.681817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.682452 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:45.684411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:45.769152 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1259/2547] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:45 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:45.773588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:45.786883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9131ad-e407-4331-97f0-34104d18d3f7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:45.977574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:45.978182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:45.978455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:45.978719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:45.979386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:45.981504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.079998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1289/2548] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:45 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:46.085948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.097322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9202dd-47f5-4ffb-a7b8-a5433e7c1a9f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.102045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.102678 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.102956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.103265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.103911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.105562 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.111519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.116951 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1260/2549] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:46.121231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.131456 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2cdae-864d-45bf-933c-24ace21d4a7a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.136181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.136773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.137051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.137322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.137960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.140177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cdeab-018c-4d6a-b138-2bb5d21abd59 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.150351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1290/2550] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:46.154499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.166338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f9c2a-d2c1-47aa-9bda-6a0ef2246489 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.172806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.173479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.173740 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.174011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] PATH_INFO: `/v3/projects/58f923d3bbfc4fbdb0e003086cdc9a6c/users/043ab3cf29fd4a798a776b405e695c6b/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.197587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58f923d3bbfc4fbdb0e003086cdc9a6c, user_id=043ab3cf29fd4a798a776b405e695c6b, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.201011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.269483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1261/2551] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:46 2022] PUT /identity/v3/projects/58f923d3bbfc4fbdb0e003086cdc9a6c/users/043ab3cf29fd4a798a776b405e695c6b/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:46.279556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.280118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.281351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.290410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:46.310985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] MFA Rules not processed for user `043ab3cf29fd4a798a776b405e695c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:46.312251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.379304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1291/2552] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:46.384383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.454340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c377b-a1c8-4dbe-a1d9-ea31ff1c07f3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.460974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.462272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.462750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.463256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.464377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.467731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.583351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1262/2553] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:46.585397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.601723 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84335e-37d6-4e02-8160-a2da6290b936 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.674756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.677041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.697707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.698309 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.698680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.699067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.699829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.702188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.754942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad504e0-9e21-41ac-bd86-1cf3d4504b59 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.759472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.760232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.760595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.760944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.762723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:46.778110 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.779640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.789798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1263/2554] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:46.822990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1292/2555] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:46 2022] POST /identity/v3/users => generated 298 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:46.828686 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.839018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508fecc-98e5-4bcc-a441-5630b7c96b28 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.844135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.849261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.849261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.849261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.849261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.849261 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.859117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1264/2556] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:46.863763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:46.874817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1581a1b-c0ff-447a-a34f-78467d8fc3d7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:46.879931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:46.886122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:46.886122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:46.886122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] PATH_INFO: `/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584/users/69335d60481f4dd0a22b09c671812bff/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:46.903985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2625a7b9c8cf4b1b83c2ab2dc6ed2584, user_id=69335d60481f4dd0a22b09c671812bff, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:46.906777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:46.989466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1293/2557] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:46 2022] PUT /identity/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584/users/69335d60481f4dd0a22b09c671812bff/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:46.993738 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.008086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f430f-a39d-4b20-95b8-153fde11acc9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.086205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:47.095827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.096581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.096861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.097131 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.099610 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.109235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22b47e-6fe1-4982-ba2b-81332c3538a0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.113839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1265/2558] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:47.117987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.128837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc5664-cdcd-4004-b675-ad7ae1a40d0d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.134053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.134806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.135106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.141689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] PATH_INFO: `/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584/users/69335d60481f4dd0a22b09c671812bff/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.141689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2625a7b9c8cf4b1b83c2ab2dc6ed2584, user_id=69335d60481f4dd0a22b09c671812bff, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.141689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.151942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1294/2559] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:47 2022] PUT /identity/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584/users/69335d60481f4dd0a22b09c671812bff/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:47.219673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.220220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.220722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.251033 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] MFA Rules not processed for user `69335d60481f4dd0a22b09c671812bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:47.252317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.334390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1266/2560] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:47.343906 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:47.346066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:47.413653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:47.422939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bbf78-76f9-4dc1-93c0-54a6a276a3c6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:47.426889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:47.427584 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:47.427862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:47.428130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:47.488105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:47.490163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:47.500809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1295/2561] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:49.932834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:49.937333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:50.009389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339f751-f110-4aaf-b7ee-a9c5414a0697 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:50.013006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:50.013614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:50.013894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:50.014252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:50.016810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:50.122126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:50.123992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:50.139228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1267/2562] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:51.582127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:51.584472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:51.603288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1a394-64f9-44ed-8fe5-05b426645a95 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:51.610176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:51.610815 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:51.611333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:51.611708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] PATH_INFO: `/v3/users/ab4df1724851433889a519583a5333b3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:51.615189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4df1724851433889a519583a5333b3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:51.615430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:51.855015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4df1724851433889a519583a5333b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:51.947245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4df1724851433889a519583a5333b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:51.949576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4df1724851433889a519583a5333b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:51.966715 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4df1724851433889a519583a5333b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:51.968048 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4e4811f2-85fe-4256-bea8-a5ac72af4d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4df1724851433889a519583a5333b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:51.993510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1296/2563] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:51 2022] DELETE /identity/v3/users/ab4df1724851433889a519583a5333b3 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:52.371481 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:52.385270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9424161d-0937-4534-88cd-744bc8bb8b9a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:52.406182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:52.444129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:52.444769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.445057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.445480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] PATH_INFO: `/v3/projects/b14e9793530a41bbab5bd4b3caecdb3c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.454294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14e9793530a41bbab5bd4b3caecdb3c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:52.454294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:52.619375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14e9793530a41bbab5bd4b3caecdb3c', 'request_id': 'req-a986a19e-fdbf-440c-9178-f768eb791e2b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:52.627762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14e9793530a41bbab5bd4b3caecdb3c', 'request_id': 'req-a986a19e-fdbf-440c-9178-f768eb791e2b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:52.707202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14e9793530a41bbab5bd4b3caecdb3c', 'request_id': 'req-a986a19e-fdbf-440c-9178-f768eb791e2b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:52.721702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] The token cache is being invalidate because project b14e9793530a41bbab5bd4b3caecdb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:52.721999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:52.722716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a986a19e-fdbf-440c-9178-f768eb791e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:52.724716 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1268/2564] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:52 2022] DELETE /identity/v3/projects/b14e9793530a41bbab5bd4b3caecdb3c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:52.729613 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.730106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.730232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.751518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:52.752764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:52.820845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1297/2565] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:52.828325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.829574 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.832722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.872966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:52.874733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:52.893373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1269/2566] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:52.898382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:52.911246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50971123-5fb9-47b9-8b69-9e366553cd69 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:52.918093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:52.924646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:52.933553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:52.945454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:52.952722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1298/2567] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:52.956975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f4c39-9f92-4434-a287-0fa52829adce None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:52.965319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:52.965319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:52.965319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:52.965319 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:52.965757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:52.968820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.110167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1270/2568] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:52 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:53.117515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.129926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb7ba0-f59e-4d79-b19b-f053a8746a9d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:53.194743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:53.195386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.195666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.195938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.196596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:53.198522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.331185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1299/2569] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:53 2022] POST /identity/v3/users => generated 283 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:53.336411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.345602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da8e89-3806-4702-b0b9-51cc9e4ec68f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:53.350297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:53.350895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.351195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.351471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.352150 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:53.353747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.365267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1271/2570] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:53.369626 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.381372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7aede-7716-4258-b15a-a8d7fe759495 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:53.386131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:53.386735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.387019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.387323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.388354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:53.390229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6995ea-be44-4d29-bf75-af98ceda7722 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.403855 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1300/2571] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:53.408499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.418894 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b937f-40de-4786-b685-57b48e9e5344 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:53.424091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:53.424847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.425207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.425479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] PATH_INFO: `/v3/projects/a41278915a254849aaf67fad76cda02d/users/1791a8a609ff4afd95e1f8d3ff7022f1/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.442269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a41278915a254849aaf67fad76cda02d, user_id=1791a8a609ff4afd95e1f8d3ff7022f1, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:53.445001 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.561664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1272/2572] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:53 2022] PUT /identity/v3/projects/a41278915a254849aaf67fad76cda02d/users/1791a8a609ff4afd95e1f8d3ff7022f1/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:53.573273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.573720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.578879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.615333 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] MFA Rules not processed for user `1791a8a609ff4afd95e1f8d3ff7022f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:53.616124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:53.696425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1301/2573] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:53.700806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.764063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:53.781835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c529e9-dfc4-48c4-9705-9d7a02736260 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:53.788208 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:53.788828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:53.790994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:53.790994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:53.790994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:53.792411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:53.934526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1273/2574] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:53 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:53.935849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:53.946814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856bc0d-c46a-4ffe-89a0-ae0cdadaad83 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.005911 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.006690 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.006969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.007350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.007988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.010346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.132715 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.134475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:54.193324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:54.203104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1302/2575] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:53 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:45:54.203792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.223796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dc04e8-5248-46cd-b0ee-a25d6f78e6e0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.233468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf520283-f43a-42aa-b3a3-5e3102034aad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.235942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.236583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.236900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.237234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.237905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.240118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.256349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.257204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.257472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.257757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.263108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1303/2576] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:54.265985 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:54.267990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.285640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483ea4c-9640-4bc1-b759-108b0f92f0a1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.298889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.299939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.303047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.303047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] PATH_INFO: `/v3/projects/a401d5534a17419aa77c8ee7aa387a9e/users/01ead60870fb4dfea5a608db3e532fc0/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.349938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a401d5534a17419aa77c8ee7aa387a9e, user_id=01ead60870fb4dfea5a608db3e532fc0, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.355948 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.366400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.367957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.396709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1274/2577] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:54.433564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1304/2578] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:54 2022] PUT /identity/v3/projects/a401d5534a17419aa77c8ee7aa387a9e/users/01ead60870fb4dfea5a608db3e532fc0/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:54.437868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.453382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444fd3b-a255-4929-bca6-a3e65cf6d26b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.556852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.558644 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.559008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.559478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.560721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.564125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5a60a-9988-430a-a615-bb3bb3903b9e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.580771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1275/2579] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:54 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:54.585818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.599669 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972bafd-96cd-4991-8416-663405fd8234 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.605088 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.606086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.606652 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.607839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] PATH_INFO: `/v3/projects/a401d5534a17419aa77c8ee7aa387a9e/users/01ead60870fb4dfea5a608db3e532fc0/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.612507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a401d5534a17419aa77c8ee7aa387a9e, user_id=01ead60870fb4dfea5a608db3e532fc0, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.616229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.633308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1305/2580] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:54 2022] PUT /identity/v3/projects/a401d5534a17419aa77c8ee7aa387a9e/users/01ead60870fb4dfea5a608db3e532fc0/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:54.648195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.648413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.648926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.683420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] MFA Rules not processed for user `01ead60870fb4dfea5a608db3e532fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:54.684587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:54.697006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:54.787646 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1276/2581] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:54.806466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:54.808939 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:54.915581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a4d6-aa87-4542-99bd-cfa7a78f61f1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:54.920838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:54.921469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:54.921744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:54.922362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:54.961641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:54.963942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:54.979858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1306/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:55.188320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:55.190301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:55.266907 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd896f-6207-44ea-8e2c-441310ff0a64 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:55.271949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:55.272593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:55.272868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:55.273138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] PATH_INFO: `/v3/users/8a45ee95cf5947c49584ff403facaff7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:55.274680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a45ee95cf5947c49584ff403facaff7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:55.276655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:55.619704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a45ee95cf5947c49584ff403facaff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:55.620748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a45ee95cf5947c49584ff403facaff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:55.802911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a45ee95cf5947c49584ff403facaff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:55.813593 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a45ee95cf5947c49584ff403facaff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:55.831326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a45ee95cf5947c49584ff403facaff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:55.838111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1277/2583] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:55 2022] DELETE /identity/v3/users/8a45ee95cf5947c49584ff403facaff7 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:56.115799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.130671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b175c6-74e1-4d0c-94b7-de873a1bf25d service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:56.196240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:56.196943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.197383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.197664 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] PATH_INFO: `/v3/projects/a994270b8e1f43328b827696f9d89bb0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.198878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a994270b8e1f43328b827696f9d89bb0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:56.200833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:56.338319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a994270b8e1f43328b827696f9d89bb0', 'request_id': 'req-bcd390f8-e090-4301-8440-62fa91ce4a8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.344960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a994270b8e1f43328b827696f9d89bb0', 'request_id': 'req-bcd390f8-e090-4301-8440-62fa91ce4a8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.346680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:56.385403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.387042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.401480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cae46-1df7-46d9-a02f-a3828fd2fff1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:56.426713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a994270b8e1f43328b827696f9d89bb0', 'request_id': 'req-bcd390f8-e090-4301-8440-62fa91ce4a8b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.428960 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:56.440417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] The token cache is being invalidate because project a994270b8e1f43328b827696f9d89bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:56.440720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.441455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bcd390f8-e090-4301-8440-62fa91ce4a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.443047 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1307/2584] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:56 2022] DELETE /identity/v3/projects/a994270b8e1f43328b827696f9d89bb0 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:56.447944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec43-842f-44dc-b263-1c9597151f57 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.448228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec43-842f-44dc-b263-1c9597151f57 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.448502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91ec43-842f-44dc-b263-1c9597151f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.464685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:56.468754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.468754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.468754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] PATH_INFO: `/v3/users/1791a8a609ff4afd95e1f8d3ff7022f1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.468754 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1791a8a609ff4afd95e1f8d3ff7022f1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:56.469157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:56.475277 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9e91ec43-842f-44dc-b263-1c9597151f57 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:56.476501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9e91ec43-842f-44dc-b263-1c9597151f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.485817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1308/2585] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:56.493634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.493924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.494256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.514707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:56.516050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.525296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1309/2586] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:56.529518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.538959 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a84ad7-89b8-40d0-bbce-491752d4c83f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:56.543863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:56.544460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.545332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.545598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.546253 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:56.546563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:56.548356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:56.561223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1310/2587] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:45:56.565940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.577227 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3a64b-075a-4c84-9c41-7d44639c58e0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:56.582050 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:56.582627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.582902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.583202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.583831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:56.586130 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:56.700447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1791a8a609ff4afd95e1f8d3ff7022f1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.701799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1791a8a609ff4afd95e1f8d3ff7022f1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.788197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1311/2588] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/projects => generated 375 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:56.799590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.801364 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.820406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1791a8a609ff4afd95e1f8d3ff7022f1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.830743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1791a8a609ff4afd95e1f8d3ff7022f1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.852630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecfcaf3c-8cce-4e1a-9e18-fef6b7dbde90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1791a8a609ff4afd95e1f8d3ff7022f1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:56.867924 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1278/2589] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:56 2022] DELETE /identity/v3/users/1791a8a609ff4afd95e1f8d3ff7022f1 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:56.872460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.873617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.874109 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.893795 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef7dfb-551f-4cbd-8b92-54cef749f54c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:56.894624 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:56.898123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:56.959560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:56.960165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:56.960439 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:56.960705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:56.961330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:56.962538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1279/2590] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:56.963345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:56.972482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:56.974047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.035824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a77d0f-01c5-4b8b-9fe3-f6b1721d8053 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.039550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.040158 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.044210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.044210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.049918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.051384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.061474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1280/2591] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:57.070787 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1312/2592] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:45:56 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:45:57.072025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.082854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341988a-02e3-48f8-ac5c-5f10ec07e238 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.087723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.091439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.091942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.092794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.093374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.095282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.107030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1281/2593] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:57.113072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.126399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a70def-1cb0-4f22-bb21-0e3172c38c6e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.132018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.132811 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.133087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.133354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.133993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.135717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad8b31-d949-41e2-89cd-b96f992b43b7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.147721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1313/2594] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:45:57.152030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.162345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362dc8-cbf0-4035-8c8f-de7316efd4da admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.167183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.167842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.168119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.168390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] PATH_INFO: `/v3/projects/c58fed2359504c93af3e64367b514912/users/f749da2333cb403e9d1f8f72670e2295/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.184398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58fed2359504c93af3e64367b514912, user_id=f749da2333cb403e9d1f8f72670e2295, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.187773 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.274186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1282/2595] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:45:57 2022] PUT /identity/v3/projects/c58fed2359504c93af3e64367b514912/users/f749da2333cb403e9d1f8f72670e2295/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:57.285534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.288577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.288577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.321157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] MFA Rules not processed for user `f749da2333cb403e9d1f8f72670e2295`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:57.323271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.399027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1314/2596] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:57.411803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.413959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.505374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244b728-f510-4ca7-9441-ee81453def3b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.511263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.511879 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.512155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.512477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.521058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.522545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.532137 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1283/2597] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:57.595722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.597423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:57.725107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932bf149-6389-4ffc-b56e-6ad66eb50354 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.730177 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.730963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.731356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.736927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] PATH_INFO: `/v3/projects/a41278915a254849aaf67fad76cda02d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.736927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a41278915a254849aaf67fad76cda02d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.736927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.824416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:57.832494 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:57.835725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde7c331-9946-4110-a744-5f743d0c7b50 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:57.840821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:57.842045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:57.842698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:57.842983 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] PATH_INFO: `/v3/users/f749da2333cb403e9d1f8f72670e2295` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:57.844181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f749da2333cb403e9d1f8f72670e2295)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:57.846160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:57.962710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a41278915a254849aaf67fad76cda02d', 'request_id': 'req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:57.972225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a41278915a254849aaf67fad76cda02d', 'request_id': 'req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.094671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a41278915a254849aaf67fad76cda02d', 'request_id': 'req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.095890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:58.107839 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] The token cache is being invalidate because project a41278915a254849aaf67fad76cda02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:58.108234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.108917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.110694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1315/2598] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:57 2022] DELETE /identity/v3/projects/a41278915a254849aaf67fad76cda02d => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:58.114927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:58.116822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:58.147512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f749da2333cb403e9d1f8f72670e2295'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.148217 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f749da2333cb403e9d1f8f72670e2295'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.192892 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc2b0a-c03e-43fc-a84f-69c539f1875b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:58.198023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:58.198671 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:58.199000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:58.199275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] PATH_INFO: `/v3/users/01ead60870fb4dfea5a608db3e532fc0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:58.200743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ead60870fb4dfea5a608db3e532fc0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:58.203605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:58.245326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f749da2333cb403e9d1f8f72670e2295'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.260082 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f749da2333cb403e9d1f8f72670e2295'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.276440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f749da2333cb403e9d1f8f72670e2295'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.285062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1284/2599] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:57 2022] DELETE /identity/v3/users/f749da2333cb403e9d1f8f72670e2295 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:45:58.286291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:58.287761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:58.353468 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d0886e-b8db-4923-933e-b2079efd940e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:58.357181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:58.357952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:58.358254 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:58.358528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:58.360352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:58.427409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:58.428868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8585c-9604-4db0-9ce8-73cdc04417aa service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:58.440794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1285/2600] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:45:58.442218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:58.442723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:58.442965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:58.463946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:58.465384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:58.481385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ead60870fb4dfea5a608db3e532fc0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.532065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1286/2601] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:58.544345 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:58.551542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ead60870fb4dfea5a608db3e532fc0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.552206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ead60870fb4dfea5a608db3e532fc0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.554888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa91b9-53a2-493a-b793-3f67bd8513b5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:58.558718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:58.559368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:58.559640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:58.559913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:58.569169 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ead60870fb4dfea5a608db3e532fc0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.570024 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ead60870fb4dfea5a608db3e532fc0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:58.572003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:58.573563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:58.584904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1316/2602] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:45:58 2022] DELETE /identity/v3/users/01ead60870fb4dfea5a608db3e532fc0 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:58.588514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1287/2603] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:45:59.295706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.297245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.306689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.309779 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.330105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75f51-9287-42dc-a7a7-e9e0aa584a99 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.418677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13d3a-ac90-41ca-a889-be4ffbc8ba4a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.425351 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.426023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.426414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.426813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] PATH_INFO: `/v3/projects/c58fed2359504c93af3e64367b514912` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.428441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58fed2359504c93af3e64367b514912)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.439550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.440291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.440627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.440903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] PATH_INFO: `/v3/projects/a401d5534a17419aa77c8ee7aa387a9e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.441580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.442286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a401d5534a17419aa77c8ee7aa387a9e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.444484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.569620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58fed2359504c93af3e64367b514912', 'request_id': 'req-66dc3307-546c-4252-a8d0-040e51800d76'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.571429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a401d5534a17419aa77c8ee7aa387a9e', 'request_id': 'req-768d5590-a6cc-4aa3-bf95-96708f9a54fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.578655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58fed2359504c93af3e64367b514912', 'request_id': 'req-66dc3307-546c-4252-a8d0-040e51800d76'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.579603 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a401d5534a17419aa77c8ee7aa387a9e', 'request_id': 'req-768d5590-a6cc-4aa3-bf95-96708f9a54fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.638957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a401d5534a17419aa77c8ee7aa387a9e', 'request_id': 'req-768d5590-a6cc-4aa3-bf95-96708f9a54fd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.650126 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:45:59.665018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] The token cache is being invalidate because project a401d5534a17419aa77c8ee7aa387a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:59.665260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.666997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-768d5590-a6cc-4aa3-bf95-96708f9a54fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.670019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1288/2604] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:59 2022] DELETE /identity/v3/projects/a401d5534a17419aa77c8ee7aa387a9e => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:59.678673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474b0e-7a71-4928-a30c-720826c3bf10 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.679058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474b0e-7a71-4928-a30c-720826c3bf10 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.679568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474b0e-7a71-4928-a30c-720826c3bf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.690069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58fed2359504c93af3e64367b514912', 'request_id': 'req-66dc3307-546c-4252-a8d0-040e51800d76'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.704999 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-36474b0e-7a71-4928-a30c-720826c3bf10 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:59.706231 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-36474b0e-7a71-4928-a30c-720826c3bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.710440 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] The token cache is being invalidate because project c58fed2359504c93af3e64367b514912 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:45:59.711103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.712156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-66dc3307-546c-4252-a8d0-040e51800d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:45:59.715006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1317/2605] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:45:59 2022] DELETE /identity/v3/projects/c58fed2359504c93af3e64367b514912 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:45:59.724042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b826-f892-4b8d-aa45-b0839ed5333e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.728550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b826-f892-4b8d-aa45-b0839ed5333e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.728550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6b826-f892-4b8d-aa45-b0839ed5333e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.757193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ccd6b826-f892-4b8d-aa45-b0839ed5333e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:59.758825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ccd6b826-f892-4b8d-aa45-b0839ed5333e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.795185 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1289/2606] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:59.803698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.804584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.805341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.832519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:59.833764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.850167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1318/2607] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:45:59.852272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.852923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.853208 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.855274 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1290/2608] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:59.859477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.869475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb7b453-756f-487b-9218-d08277bed6c8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.875329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.875962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:45:59.876085 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.876280 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.876509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.877171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:59.877275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:45:59.877474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.879111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.886450 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1319/2609] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:45:59.891759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.893770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1291/2610] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:45:59.898107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.903628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea94bf-bbfe-4a80-a706-9f1c2247e789 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.909374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.912278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.912278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.912491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba80af6-b076-404c-8518-a90e16738822 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.912604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.912604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:45:59.912604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.913710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.916698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.917367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.917666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.917866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.918521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.920908 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.931311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1320/2611] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:45:59.932350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:45:59.945380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1e050-d51b-4aa9-af35-2c5f8bac028d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:45:59.950257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:45:59.951576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:45:59.952002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:45:59.952449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:45:59.953105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:45:59.955040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:45:59.998713 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:00.028388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1292/2612] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/projects => generated 357 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:00.032656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.052606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1321/2613] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:45:59 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:00.056870 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.116794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40383407-f44f-42cf-9a46-32e7c720c22f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.148155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc003a73-2397-425c-893b-4881e2c2acc2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.153394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.154142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.154622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.154958 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.155824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.158293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.195260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.195800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.196439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.196650 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.197445 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.199517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.318753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1322/2614] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:00.323282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.333542 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b36f-016b-4b2c-ae34-f6d5a42604b7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.338320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.338917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.339226 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.339497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.340132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.342054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.352595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1323/2615] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:00.357872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.365203 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1293/2616] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/users => generated 279 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:00.369368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.372302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3250ed1-ba2d-49ec-ace9-50e0813f16b1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.377705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.378327 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.378601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.378840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.380423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.382404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8d7dd-943d-4e5e-84f8-1d2ef38bbb21 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.382981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.390220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.391212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.391810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.392282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.394371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.397268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1324/2617] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:00.403811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.406079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.414962 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1294/2618] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:00.416761 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.421550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d922-4f41-49a4-8843-f38cdce13573 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.427660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.428220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.428489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.428760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] PATH_INFO: `/v3/projects/e52831dcee864e058cee5b753b2fe4f5/users/6a47c6c0cce74c47b5ec446aff3a0799/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.432477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac227267-e3da-483a-ae8c-3ed349ac029b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.439510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.440608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.441000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.441387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.442226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.445049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.450695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52831dcee864e058cee5b753b2fe4f5, user_id=6a47c6c0cce74c47b5ec446aff3a0799, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.454875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c5922-3666-483e-a789-074f68f53f13 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.462850 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1295/2619] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:00.467433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.479354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c216b-064d-45b0-a87f-22de20cd3220 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.486095 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.486866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.487178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.487477 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] PATH_INFO: `/v3/projects/e2b2c26ca0874f7cb03be64b728c08d6/users/195cd025958846aa96bcffffd1abded1/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.512495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b2c26ca0874f7cb03be64b728c08d6, user_id=195cd025958846aa96bcffffd1abded1, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.516114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a10bf-fa2c-4f59-aa6f-03afc18b985a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.633179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1325/2620] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:00 2022] PUT /identity/v3/projects/e52831dcee864e058cee5b753b2fe4f5/users/6a47c6c0cce74c47b5ec446aff3a0799/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:00.643826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.644150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.644491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.681763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] MFA Rules not processed for user `6a47c6c0cce74c47b5ec446aff3a0799`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:00.683074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:00.683942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1296/2621] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:00 2022] PUT /identity/v3/projects/e2b2c26ca0874f7cb03be64b728c08d6/users/195cd025958846aa96bcffffd1abded1/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:00.695119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.695440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.695711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.733628 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] MFA Rules not processed for user `195cd025958846aa96bcffffd1abded1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:00.734966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:00.777581 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1326/2622] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:00.783015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.813069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1297/2623] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:00.817982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:00.872239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2047be6-7d43-4d56-8471-ade21febaf79 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.877643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.878461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.878739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.879014 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.879761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.885121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:00.901425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff79cb-dd00-47bc-82e7-8e6c64c934ca None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:00.911395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:00.912577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:00.913324 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:00.913586 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:00.914224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:00.916272 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.059357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1327/2624] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/projects => generated 365 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:01.062040 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1298/2625] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:00 2022] POST /identity/v3/projects => generated 357 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:01.080259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.084768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.168077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc09d1f-a77b-4fd1-a1d6-48b35e1b0490 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.173381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.174015 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.174291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.174563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.175331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.177670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.182972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d0362-eb5a-43b7-8d28-507da6a3af5c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.186711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.195747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.196211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.196611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.197400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.200174 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.277805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1299/2626] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:01.283055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.298595 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ebc51-5279-43f6-be61-132b13b610d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.304545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.305421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.305853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.306299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.307123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.308925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.321067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1300/2627] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:01.325395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.339460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc07f404-f8d5-4a68-b6da-7dff28235854 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.342199 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1328/2628] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:01.346278 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.347193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.347400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.347744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] PATH_INFO: `/v3/projects/f0031b1119dc495aa34a8a840df04d16/users/7e389459c8e348f29c8491aee546d0ae/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.348397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.359848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0f5ff-9a3c-4151-bc70-ba732abc21a3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.364777 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.365377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.365659 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.365934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.366597 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.368601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.371568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0031b1119dc495aa34a8a840df04d16, user_id=7e389459c8e348f29c8491aee546d0ae, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.376512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.381942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1329/2629] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:01.388637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.399464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58df53b5-afe2-42a1-821d-4767f3cf1e48 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.404476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.405146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.405428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.405701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] PATH_INFO: `/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1/users/f5d92bb426414b199402e03d26612dca/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.423093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78fac1302ff4ef8acfda7717b8a7fc1, user_id=f5d92bb426414b199402e03d26612dca, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.425794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.478912 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1301/2630] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:01 2022] PUT /identity/v3/projects/f0031b1119dc495aa34a8a840df04d16/users/7e389459c8e348f29c8491aee546d0ae/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:01.483456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.530384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1330/2631] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:01 2022] PUT /identity/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1/users/f5d92bb426414b199402e03d26612dca/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:01.534710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.589097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be515efa-3df1-44d4-a957-25efebd9a418 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.625462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdfb05-56d4-410f-a21c-3d1b57fc147d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.633172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.634803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.635295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.635724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.640369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.640369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.655949 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1331/2632] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:01.661829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.675183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bfba1-3566-4b24-bc14-c2cd04805c69 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.682057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.682768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.683669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.684101 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.684249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.684631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.684933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.685645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] PATH_INFO: `/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1/users/f5d92bb426414b199402e03d26612dca/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.686380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.688398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.689359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d78fac1302ff4ef8acfda7717b8a7fc1, user_id=f5d92bb426414b199402e03d26612dca, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.693077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7691b-bff1-4bbd-9526-ac04b3baff89 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.701814 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1302/2633] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:01.706907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1332/2634] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:01 2022] PUT /identity/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1/users/f5d92bb426414b199402e03d26612dca/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:01.708770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.719805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.720093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.720316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15320b78-8307-415c-9925-b047561c3812 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.720444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.752773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:01.760523 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] MFA Rules not processed for user `f5d92bb426414b199402e03d26612dca`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:01.761755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.812690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:01.813725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.814416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.814842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] PATH_INFO: `/v3/projects/f0031b1119dc495aa34a8a840df04d16/users/7e389459c8e348f29c8491aee546d0ae/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.817065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0031b1119dc495aa34a8a840df04d16, user_id=7e389459c8e348f29c8491aee546d0ae, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:01.820454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70d175-9924-4d66-b914-de8ddc04d08e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:01.833711 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1303/2635] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:01 2022] PUT /identity/v3/projects/f0031b1119dc495aa34a8a840df04d16/users/7e389459c8e348f29c8491aee546d0ae/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:01.846435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:01.846988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:01.847236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:01.865233 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1333/2636] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:01.877699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.879092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.883479 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] MFA Rules not processed for user `7e389459c8e348f29c8491aee546d0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:01.884698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:01.945156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41105cd3-b6b2-453c-8e50-c8308550d778 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:01.963966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1304/2637] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:01.976670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:01.978112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:02.007924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.008790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.008790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.009077 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.037666 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:02.043913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51441c8e-f11d-4f89-befb-7794fc427192 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.054335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.054335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.054335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.054335 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.059528 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.061677 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.072391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1305/2638] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:02.077129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.078578 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.096276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1334/2639] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:02.303659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.305270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:02.337031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.345883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0892e1-8470-415d-8db1-0aaf866f25ab service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.349575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.350244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.350595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.350962 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.376707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b452d-1829-475b-bdc0-d0f9719b476a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.381851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.382493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.382771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.383042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] PATH_INFO: `/v3/users/043ab3cf29fd4a798a776b405e695c6b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.384553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=043ab3cf29fd4a798a776b405e695c6b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.386895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.435951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.435951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.436605 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1335/2640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:02.437538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.446414 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123aee-7eee-4f59-8210-c17c8fbb56d1 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.451144 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.451717 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.451997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.452270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.506833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:02.514899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.518810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5d751-5a2f-4936-8b00-0534a2f25d70 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.527788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1336/2641] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:02.600585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043ab3cf29fd4a798a776b405e695c6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:02.601246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043ab3cf29fd4a798a776b405e695c6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:02.682604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043ab3cf29fd4a798a776b405e695c6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:02.690915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043ab3cf29fd4a798a776b405e695c6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:02.702431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043ab3cf29fd4a798a776b405e695c6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:02.708326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1306/2642] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:02 2022] DELETE /identity/v3/users/043ab3cf29fd4a798a776b405e695c6b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:02.713093 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.713374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.713649 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.733639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:02.734852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:02.800001 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1337/2643] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:02.810724 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:02.812384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:02.870215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430de91e-7658-479c-8c90-230dbbe6fa1e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:02.874484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:02.875516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:02.875967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:02.876410 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:02.887423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:02.889270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:02.898898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1307/2644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:02.969013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:03.374660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:03.380323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:03.391276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a443679-3af5-42e5-a7b3-54883250c159 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:03.396481 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:03.397116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:03.397389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:03.397654 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] PATH_INFO: `/v3/projects/58f923d3bbfc4fbdb0e003086cdc9a6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:03.398905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58f923d3bbfc4fbdb0e003086cdc9a6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:03.401030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:03.500748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58f923d3bbfc4fbdb0e003086cdc9a6c', 'request_id': 'req-58d381ae-8db5-4620-ba3a-fb6a18ca8525'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.507218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58f923d3bbfc4fbdb0e003086cdc9a6c', 'request_id': 'req-58d381ae-8db5-4620-ba3a-fb6a18ca8525'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.643707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58f923d3bbfc4fbdb0e003086cdc9a6c', 'request_id': 'req-58d381ae-8db5-4620-ba3a-fb6a18ca8525'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.644466 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:03.657982 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] The token cache is being invalidate because project 58f923d3bbfc4fbdb0e003086cdc9a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:03.658274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.659015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.660873 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1338/2645] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:03 2022] DELETE /identity/v3/projects/58f923d3bbfc4fbdb0e003086cdc9a6c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:03.665199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:03.680475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b26b8d-509c-4827-9aa3-b27eb60a4e35 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:03.739921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:03.740652 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:03.740931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:03.741200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] PATH_INFO: `/v3/users/69335d60481f4dd0a22b09c671812bff` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:03.742352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69335d60481f4dd0a22b09c671812bff)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:03.744421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:03.950194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69335d60481f4dd0a22b09c671812bff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:03.951492 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69335d60481f4dd0a22b09c671812bff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.015358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.071776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69335d60481f4dd0a22b09c671812bff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.081286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69335d60481f4dd0a22b09c671812bff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.095300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.097457 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3bcd087f-1b64-4383-8544-e4100d569fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69335d60481f4dd0a22b09c671812bff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.103653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1308/2646] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:03 2022] DELETE /identity/v3/users/69335d60481f4dd0a22b09c671812bff => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:04.337382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.390151 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.577424 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:04.578968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:04.603085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d381ae-8db5-4620-ba3a-fb6a18ca8525 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:04.616643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.657365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.676412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:04.677391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:04.677841 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:04.679650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:04.680270 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] PATH_INFO: `/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:04.682293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2625a7b9c8cf4b1b83c2ab2dc6ed2584)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:04.684786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:04.837689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2625a7b9c8cf4b1b83c2ab2dc6ed2584', 'request_id': 'req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.845843 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2625a7b9c8cf4b1b83c2ab2dc6ed2584', 'request_id': 'req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.923927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2625a7b9c8cf4b1b83c2ab2dc6ed2584', 'request_id': 'req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.935840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] The token cache is being invalidate because project 2625a7b9c8cf4b1b83c2ab2dc6ed2584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:04.936063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.936699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c189aaa0-55dd-444e-ab35-5c7a5bd2aebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:04.938182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1339/2647] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:04 2022] DELETE /identity/v3/projects/2625a7b9c8cf4b1b83c2ab2dc6ed2584 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:04.945374 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:04.945689 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:04.945964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:04.970749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:04.971994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.042279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1309/2648] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:05.051022 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.051336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.051611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.072359 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:05.073657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.086663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1340/2649] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:05.092125 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.104241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79908-577e-4512-aca7-237d0b8b4282 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.109617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.110332 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.110587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.122620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.122620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:05.122620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.122620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.125955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1310/2650] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:05.130514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.141835 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d1d476-7ba1-48ea-baba-7f2160abdac1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.146468 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.147060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.147377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.147645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.148295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.150211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.282887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1341/2651] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:05.334470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.334470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4a28-1bdf-43e7-9447-ce1f24c0e8f5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.364142 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.364661 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.364833 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.366718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.366718 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.371028 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.420461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:05.453239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1311/2652] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:05.458859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.480237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1d6b-5b71-40ce-a09d-d0ad65048cd3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.487444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.488055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.488329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.488594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.489234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.490901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.508445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1342/2653] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:05.508886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.518515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ddc38-246c-49d5-95cc-12529839d110 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.525277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.525885 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.526160 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.526429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.527087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.528795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f0ede-ca84-496c-a674-5d717051d9a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.540442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1312/2654] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:05.546501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.556437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b2e71-dbf1-4f17-bc5d-86fad151001c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.561926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.562572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.562846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.563115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] PATH_INFO: `/v3/projects/149dd19b0cf147e7b26ad558b74cf85c/users/a2c7edbbe80e477ca0bfcd3de5fa27e9/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.578875 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=149dd19b0cf147e7b26ad558b74cf85c, user_id=a2c7edbbe80e477ca0bfcd3de5fa27e9, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.581961 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.726496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1343/2655] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:05 2022] PUT /identity/v3/projects/149dd19b0cf147e7b26ad558b74cf85c/users/a2c7edbbe80e477ca0bfcd3de5fa27e9/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:05.734014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319dba-ea62-43d3-9059-2bcdf7c4860e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.734598 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319dba-ea62-43d3-9059-2bcdf7c4860e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.734781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80319dba-ea62-43d3-9059-2bcdf7c4860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.754942 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-80319dba-ea62-43d3-9059-2bcdf7c4860e None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:05.756216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-80319dba-ea62-43d3-9059-2bcdf7c4860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.832496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1313/2656] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:05.851139 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:05.853311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:05.925598 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8885-d287-4e99-8e57-3bf96e7118ed admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:05.929947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:05.930712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:05.930903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:05.931202 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:05.941753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:05.943355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:05.955258 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1344/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:07.326799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.327428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.327428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.358292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] MFA Rules not processed for user `a2c7edbbe80e477ca0bfcd3de5fa27e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:07.359617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:07.440963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:07.445510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1314/2658] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:07.594000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:07.596220 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:07.639916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:07.679638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067620c-7919-4498-a2c2-649738ecfff5 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:07.684636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:07.686028 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:07.686426 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:07.686859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:07.697186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:07.698678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae7fda3-e722-4c41-a286-11851ba4709d service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:07.709842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1345/2659] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:08.390168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.391649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.407460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a551a7-12ff-4a6f-9847-ed150d36f715 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.410664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.411458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.411739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.412012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] PATH_INFO: `/v3/users/6a47c6c0cce74c47b5ec446aff3a0799` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.413465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a47c6c0cce74c47b5ec446aff3a0799)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.415422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.653188 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a47c6c0cce74c47b5ec446aff3a0799'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.654166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a47c6c0cce74c47b5ec446aff3a0799'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.699386 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a47c6c0cce74c47b5ec446aff3a0799'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.707905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a47c6c0cce74c47b5ec446aff3a0799'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.722328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a47c6c0cce74c47b5ec446aff3a0799'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:08.728316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1315/2660] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:08 2022] DELETE /identity/v3/users/6a47c6c0cce74c47b5ec446aff3a0799 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:08.733295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.733779 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.733915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.753905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:08.755200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.820010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1346/2661] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:08.851881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:08.853347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:08.916974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6631a-1c7c-456a-b532-c33a5e448610 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:08.919472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:08.920409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:08.920791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:08.921062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:08.930695 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:08.932164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f3494-e3c6-4c45-bc7a-d8898b23ae2e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:08.943600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1316/2662] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:09.382000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.383562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.399203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70158e2-9aa5-4ce8-9392-aa8f34f1fda6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.406631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.407312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.407588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.407856 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] PATH_INFO: `/v3/projects/e52831dcee864e058cee5b753b2fe4f5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.408984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52831dcee864e058cee5b753b2fe4f5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.411806 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.531426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.532371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.532719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52831dcee864e058cee5b753b2fe4f5', 'request_id': 'req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.532886 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.544470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52831dcee864e058cee5b753b2fe4f5', 'request_id': 'req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.553271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] MFA Rules not processed for user `a2c7edbbe80e477ca0bfcd3de5fa27e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:09.554505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:09.613293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52831dcee864e058cee5b753b2fe4f5', 'request_id': 'req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.620964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1317/2663] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:09.631107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] The token cache is being invalidate because project e52831dcee864e058cee5b753b2fe4f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:09.631448 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.632142 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.636490 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1347/2664] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:09 2022] DELETE /identity/v3/projects/e52831dcee864e058cee5b753b2fe4f5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:09.638030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:09.665057 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:09.709698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf338796-3630-4b86-9f6f-9c65a9d3dd8f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:09.714921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:09.715641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:09.715918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:09.716193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] PATH_INFO: `/v3/users/f5d92bb426414b199402e03d26612dca` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:09.718092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d92bb426414b199402e03d26612dca)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:09.720202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:09.956585 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d92bb426414b199402e03d26612dca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:09.957343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d92bb426414b199402e03d26612dca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.031049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d92bb426414b199402e03d26612dca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.039877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d92bb426414b199402e03d26612dca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.052189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ea40d520-be65-4a0f-882b-3490ff87d9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d92bb426414b199402e03d26612dca'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.062856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1318/2665] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:09 2022] DELETE /identity/v3/users/f5d92bb426414b199402e03d26612dca => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:10.639165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:10.640729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:10.666365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5aa82-0cfd-4354-9f4b-ff62b8878c66 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:10.722296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:10.731993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:10.732668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.732936 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.733197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] PATH_INFO: `/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.734303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d78fac1302ff4ef8acfda7717b8a7fc1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:10.736238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:10.871848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd78fac1302ff4ef8acfda7717b8a7fc1', 'request_id': 'req-5b578e6b-be48-4e25-8b14-9ff472a40140'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.878785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd78fac1302ff4ef8acfda7717b8a7fc1', 'request_id': 'req-5b578e6b-be48-4e25-8b14-9ff472a40140'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.933764 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd78fac1302ff4ef8acfda7717b8a7fc1', 'request_id': 'req-5b578e6b-be48-4e25-8b14-9ff472a40140'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.945251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] The token cache is being invalidate because project d78fac1302ff4ef8acfda7717b8a7fc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:10.945547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.946314 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5b578e6b-be48-4e25-8b14-9ff472a40140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:10.947829 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1348/2666] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:10 2022] DELETE /identity/v3/projects/d78fac1302ff4ef8acfda7717b8a7fc1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:10.954003 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:10.954682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:10.954682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:10.974806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:10.976048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:11.035642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1319/2667] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:11.043756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.049082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.049082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.066170 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:11.067635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:11.079711 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1349/2668] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:11.084135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.094745 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f735-9401-4086-a8a7-1b2231d489f9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.099404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.099997 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.100277 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.100565 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.101207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:11.101502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.103094 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.115321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1320/2669] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:11.119682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.129436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379942d-a7d4-45f0-8fe0-2154ed05eb42 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.134507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.135212 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.135477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.135743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.136373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.138284 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.238683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1350/2670] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:11.243011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.253417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ce31a-39a0-4368-89f5-e12e61ed9700 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.282171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:11.316344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.316865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.317149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.317420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.318075 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.320056 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.403415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1321/2671] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:11.408094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.420573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f8cdf-3450-4689-8384-6e2dcef4f215 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.431631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.431631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.431631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.431631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.432005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.432005 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.442547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1351/2672] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:11.446534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.457966 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f690bb-f630-46b4-85f5-eff134bafe54 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.462934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.463360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.463829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.465157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.465157 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.466264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0f66d-09dd-4c1d-91b4-dd3e19dd1001 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.482394 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1322/2673] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:11.486660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.497707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed40d97-f6da-49a6-87a7-d7635b7c734b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.503633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.511864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.511864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.511864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] PATH_INFO: `/v3/projects/c6075c99ab1042229d5fc855b58e6ea7/users/de281bba5bcc4823b485fa95c692b135/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.522449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6075c99ab1042229d5fc855b58e6ea7, user_id=de281bba5bcc4823b485fa95c692b135, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.525686 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.603752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1352/2674] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:11 2022] PUT /identity/v3/projects/c6075c99ab1042229d5fc855b58e6ea7/users/de281bba5bcc4823b485fa95c692b135/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:11.615313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.615649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.615926 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.644792 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] MFA Rules not processed for user `de281bba5bcc4823b485fa95c692b135`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:11.646014 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:11.713051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1323/2675] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:11.719254 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.783196 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d502c1-1a54-4538-bbc1-09846096c373 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.787876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.788469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.788745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.789018 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.789643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.791590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:11.902079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1353/2676] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:11.906674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:11.921563 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7506c-62bc-4a98-968e-97795cef27c1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:11.987049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:11.987890 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:11.988257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:11.988581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:11.989377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:11.991498 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.089002 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1324/2677] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:11 2022] POST /identity/v3/users => generated 295 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:12.093182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:12.098825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:12.104411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ba34b-065c-4994-bd5a-4b77cccadb40 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:12.109336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:12.117409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.117409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.117409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.117409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:12.117409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.123599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1354/2678] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:12.128649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:12.138846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114f2c1-19c1-4c90-9407-381b7341b695 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:12.146623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:12.146623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.146623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.146623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] PATH_INFO: `/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f/users/84d56592ad2b483bba298905e083c29c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.168828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb0f8a6041c4af2a96a9ddf66b5c24f, user_id=84d56592ad2b483bba298905e083c29c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:12.172211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.261107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:12.266516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1325/2679] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:12 2022] PUT /identity/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f/users/84d56592ad2b483bba298905e083c29c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:12.270864 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:12.282636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed1aef-d3cd-407d-8dcc-b45c4023e3db admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:12.341193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:12.341897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.342182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.342455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.343108 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:12.344867 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42850346-67d6-4aa2-ba2d-5644d233afd5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.358236 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1355/2680] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:12.363010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:12.373614 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d214ea-0c20-48b1-9878-64a5e97ec913 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:12.379310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:12.379978 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.380257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.380529 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] PATH_INFO: `/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f/users/84d56592ad2b483bba298905e083c29c/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.382501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb0f8a6041c4af2a96a9ddf66b5c24f, user_id=84d56592ad2b483bba298905e083c29c, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:12.385346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.397963 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1326/2681] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:12 2022] PUT /identity/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f/users/84d56592ad2b483bba298905e083c29c/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:12.409269 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.409736 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.410164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.442868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] MFA Rules not processed for user `84d56592ad2b483bba298905e083c29c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:12.444057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:12.514411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1356/2682] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:12.527326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:12.529161 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:12.595103 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ae4e7-5032-4ebb-9cc0-6f9503c8b3b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:12.599342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:12.599986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:12.600249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:12.600520 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:12.611351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:12.612853 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:12.623812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1327/2683] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:13.682701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:13.692520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0b08-0c0a-40c0-8a18-009fdaa2add3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:13.697973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d56592ad2b483bba298905e083c29c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:13.698604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:13.698883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:13.699182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:13.700009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:13.700323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:13.701950 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:13.726123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1357/2684] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:13.728092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:13.741577 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf40aa-7e72-4115-88ee-3fe51d322f6f service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:13.761618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d56592ad2b483bba298905e083c29c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:13.761618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:13.761618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:13.762228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:13.762228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:13.762228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:13.856795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1328/2685] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:13 2022] POST /identity/v3/projects => generated 354 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:13.861434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:13.873242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834b4-0442-4bbe-a2c3-204c2f0ff9ca tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:13.935681 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d56592ad2b483bba298905e083c29c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:13.936288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:13.936568 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:13.936840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:13.937507 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:13.937879 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:13.940229 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:13.942145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:13.955371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1358/2686] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:13.997725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:14.008856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd7ca1-4f54-4ad2-982c-2e1666aadf36 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:14.013731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84d56592ad2b483bba298905e083c29c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:14.014288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:14.014564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:14.014831 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] PATH_INFO: `/v3/projects/a214c0e35327420584d2eb98ea1589ea` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:14.016693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a214c0e35327420584d2eb98ea1589ea)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:14.018630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:14.156115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a214c0e35327420584d2eb98ea1589ea', 'request_id': 'req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:14.164653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a214c0e35327420584d2eb98ea1589ea', 'request_id': 'req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:14.219457 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a214c0e35327420584d2eb98ea1589ea', 'request_id': 'req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:14.239226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] The token cache is being invalidate because project a214c0e35327420584d2eb98ea1589ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:14.239538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:14.240360 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f00046b-fb5d-43cc-9fb1-c77b569f1a3f tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:14.241974 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1329/2687] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:13 2022] DELETE /identity/v3/projects/a214c0e35327420584d2eb98ea1589ea => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:14.862987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:14.866466 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:14.973617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08c4c5-d2ff-4e59-beef-a2c37658e4a4 tempest-VolumeQuotaClassesTest-40809488 tempest-VolumeQuotaClassesTest-40809488-project-admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:15.070645 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:15.071331 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:15.071614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:15.071888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] PATH_INFO: `/v3/users/de281bba5bcc4823b485fa95c692b135` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:15.073074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de281bba5bcc4823b485fa95c692b135)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:15.075008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:15.330638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de281bba5bcc4823b485fa95c692b135'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:15.408216 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de281bba5bcc4823b485fa95c692b135'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:15.408838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de281bba5bcc4823b485fa95c692b135'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:15.424491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de281bba5bcc4823b485fa95c692b135'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:15.425118 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de281bba5bcc4823b485fa95c692b135'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:15.438761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1359/2688] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:14 2022] DELETE /identity/v3/users/de281bba5bcc4823b485fa95c692b135 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:15.443357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:15.447663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:15.447663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:15.463473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:15.464680 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:15.511037 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:15.533230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1330/2689] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:15.543356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:15.544792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:15.568056 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:15.609153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fdca6c-4c7e-4c09-9844-b33dd9764947 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:15.615294 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:15.615926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:15.616315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:15.616513 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:15.638926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:15.638926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:15.639107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1360/2690] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:16.258606 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:16.260176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:16.273328 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf940951-dbdd-49e7-98f1-ca362ca88ffc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:16.279384 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:16.280178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:16.280553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:16.280949 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] PATH_INFO: `/v3/projects/c6075c99ab1042229d5fc855b58e6ea7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:16.283124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6075c99ab1042229d5fc855b58e6ea7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:16.285239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:16.419325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6075c99ab1042229d5fc855b58e6ea7', 'request_id': 'req-a8e31e8f-bd84-416c-b518-6476ad31a934'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.429851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6075c99ab1042229d5fc855b58e6ea7', 'request_id': 'req-a8e31e8f-bd84-416c-b518-6476ad31a934'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.493069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6075c99ab1042229d5fc855b58e6ea7', 'request_id': 'req-a8e31e8f-bd84-416c-b518-6476ad31a934'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.505958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] The token cache is being invalidate because project c6075c99ab1042229d5fc855b58e6ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:16.510080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.510080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.511815 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1331/2691] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:16 2022] DELETE /identity/v3/projects/c6075c99ab1042229d5fc855b58e6ea7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:16.517127 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:16.527096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278119e-9c40-4b46-8825-7903e5ea0f01 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:16.587796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:16.588437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:16.588716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:16.588988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] PATH_INFO: `/v3/users/84d56592ad2b483bba298905e083c29c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:16.590161 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d56592ad2b483bba298905e083c29c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:16.592138 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:16.818825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d56592ad2b483bba298905e083c29c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.893561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d56592ad2b483bba298905e083c29c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.894273 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d56592ad2b483bba298905e083c29c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.908674 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d56592ad2b483bba298905e083c29c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.909587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d2d79ae7-1579-42f2-9434-7abe7bd89569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d56592ad2b483bba298905e083c29c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:16.923585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1361/2692] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:16 2022] DELETE /identity/v3/users/84d56592ad2b483bba298905e083c29c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:17.334954 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:17.336921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:17.357084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e31e8f-bd84-416c-b518-6476ad31a934 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:17.463944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:17.464640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.464920 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.465187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] PATH_INFO: `/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.466369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb0f8a6041c4af2a96a9ddf66b5c24f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:17.468559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:17.563571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'request_id': 'req-a0e65f5a-9395-40c3-be37-ec6925d6ee88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:17.569972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'request_id': 'req-a0e65f5a-9395-40c3-be37-ec6925d6ee88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:17.612929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb0f8a6041c4af2a96a9ddf66b5c24f', 'request_id': 'req-a0e65f5a-9395-40c3-be37-ec6925d6ee88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:17.624234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] The token cache is being invalidate because project 0cb0f8a6041c4af2a96a9ddf66b5c24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:17.624542 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:17.625404 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0e65f5a-9395-40c3-be37-ec6925d6ee88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:17.627066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1332/2693] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:17 2022] DELETE /identity/v3/projects/0cb0f8a6041c4af2a96a9ddf66b5c24f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:17.632631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.633143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.633143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.655668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:17.657537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:17.697230 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:17.718224 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1362/2694] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:17.727123 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.727432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.727733 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.736676 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:17.748465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:17.750402 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:17.758917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1333/2695] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:17.763393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:17.774297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308e200-86e5-431d-95d3-bbf18c5b98bc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:17.780231 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:17.780838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.781155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.781428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.782091 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:17.782394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:17.784038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:17.796487 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1363/2696] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:17.801090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:17.810512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceffdfc-c48f-4d4e-8d19-5c5b946115cf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:17.817619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:17.818239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.823670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.823670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.823670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:17.823670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:17.901317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1334/2697] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:17 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:17.905548 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:17.916102 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d46e13-a92d-44eb-ba04-233c5fa09421 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:17.975132 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:17.975685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:17.975966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:17.976243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:17.976891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:17.978822 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:18.071562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1364/2698] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:17 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:18.075827 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:18.089485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbae34-1d63-4062-b5de-b242b0d1eecb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:18.095964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:18.096816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:18.097204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:18.097712 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:18.098532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:18.101044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:18.118353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1335/2699] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:18.122625 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:18.134078 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93743b35-5abf-4b18-b8af-6a9390eac8e1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:18.138876 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:18.139563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:18.143316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:18.143316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:18.143316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:18.143316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbecfda-c3fb-47ca-851e-3d575f229128 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:18.154101 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1365/2700] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:18.158660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:18.168846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e1531-01d1-453c-9852-3072c1ec9788 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:18.174367 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:18.175025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:18.175342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:18.175618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] PATH_INFO: `/v3/projects/f72fb03d8a4442449667fe48ee3a9387/users/43a31af78d3641a19f20acb8b13c9129/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:18.192427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f72fb03d8a4442449667fe48ee3a9387, user_id=43a31af78d3641a19f20acb8b13c9129, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:18.195097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:18.299510 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1336/2701] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:18 2022] PUT /identity/v3/projects/f72fb03d8a4442449667fe48ee3a9387/users/43a31af78d3641a19f20acb8b13c9129/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:18.311315 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:18.311595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:18.311871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:18.342299 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] MFA Rules not processed for user `43a31af78d3641a19f20acb8b13c9129`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:18.344734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:18.413049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1366/2702] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:18.425913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:18.432251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:18.505651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b89ce-fa78-4ce7-aa2e-9a0a2ddf8971 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:18.515702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:18.520234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:18.520234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:18.520234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:18.527889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:18.529340 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d119b28-eff3-46c8-988a-cce45f9130c5 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:18.543299 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1337/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:22.634543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:22.685065 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:22.712644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837fb2b-4c4d-467e-8101-83725f7c6655 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:22.719457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:22.720192 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:22.720473 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:22.720746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] PATH_INFO: `/v3/users/43a31af78d3641a19f20acb8b13c9129` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:22.721951 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a31af78d3641a19f20acb8b13c9129)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:22.724008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:22.922969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a31af78d3641a19f20acb8b13c9129'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.042011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a31af78d3641a19f20acb8b13c9129'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.042629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a31af78d3641a19f20acb8b13c9129'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.057095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a31af78d3641a19f20acb8b13c9129'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.058099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a31af78d3641a19f20acb8b13c9129'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.071915 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1367/2704] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:22 2022] DELETE /identity/v3/users/43a31af78d3641a19f20acb8b13c9129 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:23.076722 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:23.077232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:23.077232 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:23.098518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:23.099789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:23.166080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1338/2705] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:23.173630 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:23.175092 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:23.243611 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3fa0-36e2-4e55-92d8-bc492636867e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:23.247752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:23.249083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:23.249083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:23.249083 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:23.265096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:23.266577 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d638b9-0536-4cf9-b844-9bcf765fb8f3 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:23.277365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1368/2706] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:23.706700 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:23.708174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:23.722039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25defb8-c761-4f49-9f10-05d95249ff8c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:23.726922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:23.727608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:23.727888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:23.728159 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] PATH_INFO: `/v3/projects/f72fb03d8a4442449667fe48ee3a9387` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:23.729306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f72fb03d8a4442449667fe48ee3a9387)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:23.731429 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:23.824483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:23.832795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f72fb03d8a4442449667fe48ee3a9387', 'request_id': 'req-df9b8583-b1c2-47a3-8133-98e8a65f79c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.839209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f72fb03d8a4442449667fe48ee3a9387', 'request_id': 'req-df9b8583-b1c2-47a3-8133-98e8a65f79c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.909459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f72fb03d8a4442449667fe48ee3a9387', 'request_id': 'req-df9b8583-b1c2-47a3-8133-98e8a65f79c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.921413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] The token cache is being invalidate because project f72fb03d8a4442449667fe48ee3a9387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:23.921669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.922320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-df9b8583-b1c2-47a3-8133-98e8a65f79c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:23.923846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1339/2707] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:23 2022] DELETE /identity/v3/projects/f72fb03d8a4442449667fe48ee3a9387 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:23.928910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:23.929488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:23.929488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:23.951680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:23.953406 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.015660 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:24.036560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1369/2708] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:24.044727 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.045079 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.045524 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.075425 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:24.080631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.086707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1340/2709] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:24.091075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.101790 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22040f-8d29-4ba4-a07b-d28d81a7ef6b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.107137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.108043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.108391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.109545 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.110297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:24.110606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.112275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.129049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1370/2710] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:24.134377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.145734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25601f66-61f7-4456-99f5-959be42b20c3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.150453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.151081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.152177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.152177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.152381 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.154200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.256534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1341/2711] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:24.258278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.269904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7765e-0bed-4dd8-96e2-c5d23504372e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.337991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.338990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.339499 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.339963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.341096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.346143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.376594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.378608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.392747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf42f5-d651-42a0-bef0-02411ff7e88c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.398313 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.399540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.399540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.399540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] PATH_INFO: `/v3/users/195cd025958846aa96bcffffd1abded1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.400470 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=195cd025958846aa96bcffffd1abded1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.402408 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.447410 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1371/2712] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:24.451889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.464423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52187e9-f35b-47e6-94a4-a669241838ea admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.469465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.470079 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.475460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.475460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.475460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.475460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.485697 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1372/2713] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:24.489785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.500276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fda084-0de9-4244-b5ae-66be86acd497 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.505423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.506076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.506349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.506615 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.512945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.512945 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.521609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1373/2714] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:24.523976 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.535318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462db5a-a2e1-4328-8c82-5db74ad85518 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.542051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.542782 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.543103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.543384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] PATH_INFO: `/v3/projects/c2824f77bc9a469991ee1c973cf64a34/users/75178b067737458f97fed479bcb66a62/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.560185 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2824f77bc9a469991ee1c973cf64a34, user_id=75178b067737458f97fed479bcb66a62, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.563362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d4c88-11e9-4be8-9fe4-b034716fc7e9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.604649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '195cd025958846aa96bcffffd1abded1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:24.605415 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '195cd025958846aa96bcffffd1abded1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:24.624737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1374/2715] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:24 2022] PUT /identity/v3/projects/c2824f77bc9a469991ee1c973cf64a34/users/75178b067737458f97fed479bcb66a62/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:24.635360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.635643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.635917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.664347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '195cd025958846aa96bcffffd1abded1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:24.665382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] MFA Rules not processed for user `75178b067737458f97fed479bcb66a62`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:24.666667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.674207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '195cd025958846aa96bcffffd1abded1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:24.690052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d461e8a-539d-4777-b3f0-c1f86d0fdd14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '195cd025958846aa96bcffffd1abded1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:24.696776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1342/2716] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:24 2022] DELETE /identity/v3/users/195cd025958846aa96bcffffd1abded1 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:24.701202 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.701490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.701866 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.723447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:24.724600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.739455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1375/2717] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:24.744540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.748214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.768232 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:24.789181 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1343/2718] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:24.799763 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.801235 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:24.815354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:24.816461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db34ad1-fe9d-460a-9936-4cd0dea09973 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.822435 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.823572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.824011 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.824462 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.825111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.827076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.864149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6d05a-f56a-471d-9fdb-46bb736343af None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:24.867965 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:24.868651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:24.869134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:24.869566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:24.879443 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:24.880921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:24.891044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1344/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:24.943809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1376/2720] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:24.947204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:24.958211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be50e45-deab-4130-bc06-37d41b95c8f2 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.017070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.017671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.018061 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.018305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.018971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.020921 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.138859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1345/2721] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:24 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:25.143373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.153973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa785a6-65c1-4ab4-9c4b-072771da43ce admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.158901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.159616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.160180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.160492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.161141 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.163235 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.174964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1377/2722] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:25.179010 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.190935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e82db1-7ccd-4c97-b4fb-f2433b48959d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.195707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.196343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.196621 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.196892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.197698 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.199405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.209568 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1346/2723] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:25.213760 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.224636 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79499b24-5f63-49f0-af65-a05898c9cc67 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.229292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.230160 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.230271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.230498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] PATH_INFO: `/v3/projects/e066418afba7455c88b091089225327f/users/64c6297bbdb54bf6bfb5699abc97e0ed/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.246990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e066418afba7455c88b091089225327f, user_id=64c6297bbdb54bf6bfb5699abc97e0ed, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.250385 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f407e35-2a6b-4e39-bbd9-78d8f3597d43 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.306102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.308044 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:25.323285 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0013ad-f02b-4d79-8dee-31b4b86d5c90 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.331351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.332046 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.332327 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.332594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] PATH_INFO: `/v3/projects/e2b2c26ca0874f7cb03be64b728c08d6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.333846 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b2c26ca0874f7cb03be64b728c08d6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.335464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1378/2724] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:25 2022] PUT /identity/v3/projects/e066418afba7455c88b091089225327f/users/64c6297bbdb54bf6bfb5699abc97e0ed/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:25.336821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.347176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.347486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.347761 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.376643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] MFA Rules not processed for user `64c6297bbdb54bf6bfb5699abc97e0ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:25.377947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:25.444186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1379/2725] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:25.444526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b2c26ca0874f7cb03be64b728c08d6', 'request_id': 'req-0e3750bf-7f03-4cc4-94c6-594960dd763f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.448689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.451670 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b2c26ca0874f7cb03be64b728c08d6', 'request_id': 'req-0e3750bf-7f03-4cc4-94c6-594960dd763f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.510024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b2c26ca0874f7cb03be64b728c08d6', 'request_id': 'req-0e3750bf-7f03-4cc4-94c6-594960dd763f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.510530 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5fa4-fc64-4e84-adc8-1d10f4b070ce None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.514336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:25.522118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] The token cache is being invalidate because project e2b2c26ca0874f7cb03be64b728c08d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:25.522414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.523070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.524631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1347/2726] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:25 2022] DELETE /identity/v3/projects/e2b2c26ca0874f7cb03be64b728c08d6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:25.528526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.529964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:25.548746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:25.572465 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.573076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.573825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.574221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.574874 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.577103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.601922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3750bf-7f03-4cc4-94c6-594960dd763f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.606784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.607456 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.607736 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.608011 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] PATH_INFO: `/v3/users/7e389459c8e348f29c8491aee546d0ae` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.609690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e389459c8e348f29c8491aee546d0ae)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.611757 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.682521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1380/2727] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:25 2022] POST /identity/v3/projects => generated 375 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:25.687547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.689415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:25.757537 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debdcb31-cdc0-444f-b0e6-06e444d84d5a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.762313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.762851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.763129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.763429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.764064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.766040 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.809778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e389459c8e348f29c8491aee546d0ae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.810433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e389459c8e348f29c8491aee546d0ae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.876075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1381/2728] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:25 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:25.880172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.881765 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:25.892541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e389459c8e348f29c8491aee546d0ae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.893952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffc7f5-ff38-4c01-886b-6b29ced9e9ff admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.898947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.899580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.899859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.903866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:25.903866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:25.904069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e389459c8e348f29c8491aee546d0ae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.904165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:25.913662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e389459c8e348f29c8491aee546d0ae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:25.914561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1382/2729] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:25.918257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:25.920302 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1348/2730] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:25 2022] DELETE /identity/v3/users/7e389459c8e348f29c8491aee546d0ae => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:25.929797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e393040-bbc9-4005-9d66-b2274c5d0edd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:25.990419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:25.991117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:25.991374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:25.991816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] PATH_INFO: `/v3/projects/ed65b8455ea046edb50da1dd2a289509/users/71bccde638fa43e4bc2b199391502535/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.008009 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed65b8455ea046edb50da1dd2a289509, user_id=71bccde638fa43e4bc2b199391502535, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.010679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.013105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:26.111800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1383/2731] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:25 2022] PUT /identity/v3/projects/ed65b8455ea046edb50da1dd2a289509/users/71bccde638fa43e4bc2b199391502535/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:26.116134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.126874 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90297dee-7ea0-41b9-a966-b65e74b191e1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.186024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.187870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.187870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.187870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.188175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.189764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e4161-6527-403e-bb44-23a2b1214c7a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.200674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1349/2732] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:26.205349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.218186 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6debfb-315b-4c0c-bb07-dcb6b4b2d8a2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.223046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.223742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.224013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.224288 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] PATH_INFO: `/v3/projects/ed65b8455ea046edb50da1dd2a289509/users/71bccde638fa43e4bc2b199391502535/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.226391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed65b8455ea046edb50da1dd2a289509, user_id=71bccde638fa43e4bc2b199391502535, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.229095 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.255312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1384/2733] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:26 2022] PUT /identity/v3/projects/ed65b8455ea046edb50da1dd2a289509/users/71bccde638fa43e4bc2b199391502535/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:26.261909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.261909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.261909 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.294215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] MFA Rules not processed for user `71bccde638fa43e4bc2b199391502535`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:26.295519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.315667 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:26.358972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.361264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.380642 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1350/2734] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:26.390392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.392310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.432967 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587b2b-37f5-4741-8b91-b560d76e424b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.438644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.479376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.479376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.479376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] PATH_INFO: `/v3/projects/f0031b1119dc495aa34a8a840df04d16` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.479376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0031b1119dc495aa34a8a840df04d16)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.479376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.479887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731d1be-7322-4b26-beca-430c23519e42 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.479887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.479887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.479887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.480282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.480282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.519132 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.520657 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfff3ad-9d87-4249-87a5-deb3c8ca72e4 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.529734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1351/2735] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:26.619242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0031b1119dc495aa34a8a840df04d16', 'request_id': 'req-508b0311-151e-4f99-9245-b0f5a1d38966'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:26.625601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0031b1119dc495aa34a8a840df04d16', 'request_id': 'req-508b0311-151e-4f99-9245-b0f5a1d38966'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:26.755360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0031b1119dc495aa34a8a840df04d16', 'request_id': 'req-508b0311-151e-4f99-9245-b0f5a1d38966'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:26.768191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] The token cache is being invalidate because project f0031b1119dc495aa34a8a840df04d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:26.768563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:26.769317 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-508b0311-151e-4f99-9245-b0f5a1d38966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:26.771116 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1385/2736] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:26 2022] DELETE /identity/v3/projects/f0031b1119dc495aa34a8a840df04d16 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:26.775899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.776189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.776685 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.796590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:26.797784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.870105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1352/2737] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:26.878985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.879283 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.879561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.900952 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:26.902155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:26.911394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1386/2738] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:26.915765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.929244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80412e95-00fc-40c6-a721-1ba2913b9a09 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.931616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.932220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.932499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.932771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.933418 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:26.933721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.935368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:26.943187 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:26.951849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1353/2739] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:26.954404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:26.964305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa1b90-a96e-4154-9d49-3dbfcbf9fcee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:26.969165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:26.969902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:26.970183 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:26.970455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:26.971090 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:26.973020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.068303 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1387/2740] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:26 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:27.072411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.083434 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ea759-f71d-407f-aaf1-24a390be429e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.159325 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.159625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.159957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.160296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.161016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.163246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.279192 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1354/2741] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:27 2022] POST /identity/v3/users => generated 282 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:27.281531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.301135 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead8d73-3c06-410d-9ee5-12831769ada4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.307076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.307752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.308034 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.308305 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.308957 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.310567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.329015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1388/2742] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:27.332708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.344437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50a4ce-aacb-4194-8a8d-cfe0b029bdd6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.350482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.350482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.351087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.351351 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.352145 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.370356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213788a3-8cde-4e1b-b759-254853d35c44 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.379107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1355/2743] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:27.383404 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.394149 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0287-46ce-4e3d-88a2-67997cc44237 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.398927 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.399616 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.399893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.400164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] PATH_INFO: `/v3/projects/e22c0f8dafda47e8880c4755a89eef96/users/657b8d8d5ac74af8bf1e2f8c598fda85/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.415800 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22c0f8dafda47e8880c4755a89eef96, user_id=657b8d8d5ac74af8bf1e2f8c598fda85, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.418527 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.534071 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1389/2744] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:27 2022] PUT /identity/v3/projects/e22c0f8dafda47e8880c4755a89eef96/users/657b8d8d5ac74af8bf1e2f8c598fda85/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:27.546664 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.546984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.547291 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.577077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] MFA Rules not processed for user `657b8d8d5ac74af8bf1e2f8c598fda85`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:27.579387 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:27.648436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1356/2745] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:27.653238 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.663809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:27.721886 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20223d-bc58-4f5f-91a6-eae0b5c2b7f4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.726943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.727586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.727866 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.728165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.728825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.730745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.848482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1390/2746] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:27 2022] POST /identity/v3/projects => generated 363 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:27.852957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:27.865108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a216e-5406-4daa-8ffa-f70bbf4a8c9d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:27.923099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:27.923747 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:27.924023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:27.924295 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:27.924934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:27.926864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:27.939289 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:28.012863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1357/2747] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:27 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:28.017437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.028318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1d1ac-cbd1-4417-a53b-e33903071650 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.033239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.036360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.036360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.036360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.036360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.036798 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.048606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1391/2748] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:28.056338 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.067961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440e166-d587-41d6-b680-a12086eb344c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.072719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.073388 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.073673 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.073972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] PATH_INFO: `/v3/projects/a50120c6986b4d94ae4d4e897f6b8800/users/bcb7a81ad9b645499f850eee164eee57/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.092437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50120c6986b4d94ae4d4e897f6b8800, user_id=bcb7a81ad9b645499f850eee164eee57, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.095693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.149849 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1358/2749] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:28 2022] PUT /identity/v3/projects/a50120c6986b4d94ae4d4e897f6b8800/users/bcb7a81ad9b645499f850eee164eee57/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:28.154044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.164328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413672-bab6-42d9-abad-ed496537fca6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.224297 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.224910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.225197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.225937 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.226100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.227720 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c24f7-253b-41f9-87d1-64039a87b227 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.241193 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1392/2750] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:28.245816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.257499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84b180-b8a7-4e88-b6ba-a5a0f044dc8c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.262505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.263276 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.263599 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.263789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] PATH_INFO: `/v3/projects/a50120c6986b4d94ae4d4e897f6b8800/users/bcb7a81ad9b645499f850eee164eee57/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.265753 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50120c6986b4d94ae4d4e897f6b8800, user_id=bcb7a81ad9b645499f850eee164eee57, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.268512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.280475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1359/2751] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:28 2022] PUT /identity/v3/projects/a50120c6986b4d94ae4d4e897f6b8800/users/bcb7a81ad9b645499f850eee164eee57/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:28.292355 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.292612 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.292885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.325486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] MFA Rules not processed for user `bcb7a81ad9b645499f850eee164eee57`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:28.326969 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:28.410793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1393/2752] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:28.423026 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.425036 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:28.490455 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72184f-2093-4df4-82a3-cf0ecaa976cb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.494427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.495051 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.495369 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.495647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.506220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.507726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.523312 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1360/2753] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:28.582905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.584403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:28.591622 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:28.598555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06511041-fa28-4a3b-b455-d11683fd7c5b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.602369 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.602977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.603295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.603569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.605423 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:28.675032 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.676522 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.686865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1394/2754] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:28.736211 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:28.745530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca3324-d762-4db3-9f13-fe97ed8d859c service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:28.750552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:28.751193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:28.751467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:28.751735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:28.828298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:28.829755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391490d-c2b5-4e86-8709-4d153bb253ab service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:28.839534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1361/2755] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:30.261632 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:30.270689 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbf2707-65be-46a6-90bb-6ea47bb698cb service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:30.275460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:30.275460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:30.275460 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:30.275786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:30.277325 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:30.343693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:30.345140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:30.354744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1395/2756] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:31.856300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d53d64-e18a-4ba6-80f5-e5b9cb31708b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:31.856844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d53d64-e18a-4ba6-80f5-e5b9cb31708b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:31.856948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d53d64-e18a-4ba6-80f5-e5b9cb31708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:31.876228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d8d53d64-e18a-4ba6-80f5-e5b9cb31708b None None] MFA Rules not processed for user `a2c7edbbe80e477ca0bfcd3de5fa27e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:31.877416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d8d53d64-e18a-4ba6-80f5-e5b9cb31708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:31.933349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1362/2757] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:32.394687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:32.465125 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e082879-5027-494a-a550-6a88e90da9ef service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:32.472682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:32.472682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:32.472682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:32.473296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] PATH_INFO: `/v3/users/657b8d8d5ac74af8bf1e2f8c598fda85` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:32.474173 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=657b8d8d5ac74af8bf1e2f8c598fda85)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:32.476157 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:32.664515 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:32.674585 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '657b8d8d5ac74af8bf1e2f8c598fda85'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:32.756339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '657b8d8d5ac74af8bf1e2f8c598fda85'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:32.757546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '657b8d8d5ac74af8bf1e2f8c598fda85'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:32.789865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '657b8d8d5ac74af8bf1e2f8c598fda85'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:32.790667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '657b8d8d5ac74af8bf1e2f8c598fda85'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:32.807893 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1396/2758] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:32 2022] DELETE /identity/v3/users/657b8d8d5ac74af8bf1e2f8c598fda85 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:32.813183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:32.813474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:32.813748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:32.822183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:32.833507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:32.834746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:32.900218 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1363/2759] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:32.910737 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:32.912182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:32.977842 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa24fd-72a4-456b-9caf-09603ad25ef4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:32.981596 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:32.982285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:32.982565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:32.983572 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:32.994042 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:32.995521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:33.006027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1397/2760] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:33.470506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:33.471961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:33.488437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea4ae2-a94f-4cd2-8467-c638267b7918 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:33.493616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:33.494354 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:33.494636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:33.494898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] PATH_INFO: `/v3/projects/e22c0f8dafda47e8880c4755a89eef96` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:33.496084 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e22c0f8dafda47e8880c4755a89eef96)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:33.498138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:33.621597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e22c0f8dafda47e8880c4755a89eef96', 'request_id': 'req-8f53db9f-5363-4cb0-b59f-eb21462be104'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:33.628350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e22c0f8dafda47e8880c4755a89eef96', 'request_id': 'req-8f53db9f-5363-4cb0-b59f-eb21462be104'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:33.732364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e22c0f8dafda47e8880c4755a89eef96', 'request_id': 'req-8f53db9f-5363-4cb0-b59f-eb21462be104'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:33.748613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] The token cache is being invalidate because project e22c0f8dafda47e8880c4755a89eef96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:33.749122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:33.749889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:33.751767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1364/2761] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:33 2022] DELETE /identity/v3/projects/e22c0f8dafda47e8880c4755a89eef96 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:33.756279 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:33.772502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11607de0-c7e8-4512-95c8-e15600a525b3 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:33.834342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:33.835128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:33.835501 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:33.835852 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] PATH_INFO: `/v3/users/bcb7a81ad9b645499f850eee164eee57` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:33.837107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb7a81ad9b645499f850eee164eee57)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:33.839459 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:33.986243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb7a81ad9b645499f850eee164eee57'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.057521 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb7a81ad9b645499f850eee164eee57'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.058155 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb7a81ad9b645499f850eee164eee57'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.064289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:34.072146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb7a81ad9b645499f850eee164eee57'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.072859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-785d27f7-f7b4-4a80-96ea-1fee37b1a17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb7a81ad9b645499f850eee164eee57'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.086175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1398/2762] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:33 2022] DELETE /identity/v3/users/bcb7a81ad9b645499f850eee164eee57 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:34.562830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:34.564400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:34.577668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53db9f-5363-4cb0-b59f-eb21462be104 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:34.656400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a669e-4405-4199-b992-74a81913232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:34.657096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a669e-4405-4199-b992-74a81913232f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.657368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a669e-4405-4199-b992-74a81913232f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.657631 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a669e-4405-4199-b992-74a81913232f admin admin] PATH_INFO: `/v3/projects/a50120c6986b4d94ae4d4e897f6b8800` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.658778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a669e-4405-4199-b992-74a81913232f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50120c6986b4d94ae4d4e897f6b8800)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:34.660795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a669e-4405-4199-b992-74a81913232f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:34.755065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:34.768430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50120c6986b4d94ae4d4e897f6b8800', 'request_id': 'req-562a669e-4405-4199-b992-74a81913232f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.774684 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50120c6986b4d94ae4d4e897f6b8800', 'request_id': 'req-562a669e-4405-4199-b992-74a81913232f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.823600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50120c6986b4d94ae4d4e897f6b8800', 'request_id': 'req-562a669e-4405-4199-b992-74a81913232f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.841467 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] The token cache is being invalidate because project a50120c6986b4d94ae4d4e897f6b8800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:34.841756 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.843344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-562a669e-4405-4199-b992-74a81913232f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:34.846847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1365/2763] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:34 2022] DELETE /identity/v3/projects/a50120c6986b4d94ae4d4e897f6b8800 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:34.853780 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.857725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.857725 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.877343 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:34.878678 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:34.957217 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1399/2764] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:34.965682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:34.965967 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:34.966919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:34.989135 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:34.992539 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.003215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1366/2765] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:35.004544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.015107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f586a5-6a70-4e38-acc6-8812a48d0bde None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.020771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.021552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.021983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.024257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.024257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:35.024257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.026582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.063365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1400/2766] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:35.063473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.064592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.067699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.093752 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf52eb-0959-4f9d-8ca5-f8ccd35e61c7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.098791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab3554-d27e-425d-b43f-3257da7caa23 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.102586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.104097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.104586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.105053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] PATH_INFO: `/v3/users/75178b067737458f97fed479bcb66a62` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.106800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.107381 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75178b067737458f97fed479bcb66a62)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.108881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.110175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.110503 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.112403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.113550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.117623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.265265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1367/2767] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/projects => generated 399 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:35.269333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.344262 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bcdc20-5d01-4ca5-b280-34db5298bbda admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.349516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.350149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.350431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.350705 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.351523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.353389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.361498 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75178b067737458f97fed479bcb66a62'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:35.437671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1368/2768] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/users => generated 300 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:35.440677 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75178b067737458f97fed479bcb66a62'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:35.441262 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75178b067737458f97fed479bcb66a62'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:35.441946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.454286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f4d2-9ed5-4455-937b-477e6da66887 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.458307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75178b067737458f97fed479bcb66a62'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:35.459089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52603605-a2f3-492a-894e-ca578b824016 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75178b067737458f97fed479bcb66a62'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:35.464321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.464321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.464321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.464321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.464791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.464791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.478380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1369/2769] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:35.478631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1401/2770] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:35 2022] DELETE /identity/v3/users/75178b067737458f97fed479bcb66a62 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:35.479961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.480564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.480857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.481152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.483829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.497243 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd68b9-74af-44fc-baf5-3f9899860f81 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.505514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:35.506770 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.544682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:35.565899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.566797 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.567096 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.567764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.568267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.570092 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.578206 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1402/2771] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:35.582546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1370/2772] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:35.586405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.592998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.597464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.600712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1f059-a6bc-4a6a-853c-38b61e3db44a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.604839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:35.610097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.610746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.611044 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.611412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] PATH_INFO: `/v3/projects/9d2be6f5420a4e8983c833ba4f3d6ee4/users/a71077ca7b1e46a1b472ea6d0c57f565/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.632657 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2be6f5420a4e8983c833ba4f3d6ee4, user_id=a71077ca7b1e46a1b472ea6d0c57f565, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.635486 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.665922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87367f7a-2a7d-4f45-bc8f-8fb621ba3e7c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.670578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.671241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.671561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.671848 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.681708 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.683440 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f208fee-d45c-44ba-b0b9-7774e91e0ed6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:35.694271 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1371/2773] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:35.721995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1403/2774] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:35 2022] PUT /identity/v3/projects/9d2be6f5420a4e8983c833ba4f3d6ee4/users/a71077ca7b1e46a1b472ea6d0c57f565/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:35.733341 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.733625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.733889 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.771513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] MFA Rules not processed for user `a71077ca7b1e46a1b472ea6d0c57f565`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:35.772781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:35.844742 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1372/2775] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:35.849318 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:35.918935 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ace62-c94b-4160-ab72-b51295d65201 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:35.925263 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:35.926983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:35.927382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:35.927900 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:35.928641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:35.930990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.025181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1404/2776] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:35 2022] POST /identity/v3/projects => generated 401 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:36.029331 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.041945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36c563-1bd3-4ac3-ab3a-0e7c315af903 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.122300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.122952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.123887 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.124464 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.126480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.127174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.229378 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1373/2777] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:36 2022] POST /identity/v3/users => generated 307 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:36.234210 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.246150 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a389b-de08-4026-8560-c8f1cb88a6a9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.253685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.255994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.255994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.255994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.255994 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.257342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.269721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1405/2778] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:36.275461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.276396 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.278134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:36.286343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f520db-0bc7-4ff6-9998-abcebcc3767f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.292582 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984563a9-17b5-4f8b-aaa5-216b7708e617 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.292811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.293638 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.294042 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.294530 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] PATH_INFO: `/v3/projects/ef8044ead34847cbbc0401f752270fe6/users/487f6b4fac53420db417a67688d8f414/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.298535 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.299215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.299511 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.299769 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] PATH_INFO: `/v3/projects/c2824f77bc9a469991ee1c973cf64a34` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.301026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2824f77bc9a469991ee1c973cf64a34)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.303012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.319116 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8044ead34847cbbc0401f752270fe6, user_id=487f6b4fac53420db417a67688d8f414, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.324979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.418076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1374/2779] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:36 2022] PUT /identity/v3/projects/ef8044ead34847cbbc0401f752270fe6/users/487f6b4fac53420db417a67688d8f414/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:36.422177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.426501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:36.457820 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2824f77bc9a469991ee1c973cf64a34', 'request_id': 'req-2af34f0e-f0b2-45dc-bab0-e2af943baadd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.465919 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2824f77bc9a469991ee1c973cf64a34', 'request_id': 'req-2af34f0e-f0b2-45dc-bab0-e2af943baadd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.494750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4f1fb-95af-4fea-be56-ff10d17fac78 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.571115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.572070 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.572395 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.572663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2824f77bc9a469991ee1c973cf64a34', 'request_id': 'req-2af34f0e-f0b2-45dc-bab0-e2af943baadd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.572787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.573317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.574972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.585887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] The token cache is being invalidate because project c2824f77bc9a469991ee1c973cf64a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:36.586250 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.586515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1375/2780] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:36.586851 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.588510 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1406/2781] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:36 2022] DELETE /identity/v3/projects/c2824f77bc9a469991ee1c973cf64a34 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:36.590265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.592931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.593107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:36.594413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:36.607078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721eb42-c974-4d66-8a70-39496cf5738a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.610567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af34f0e-f0b2-45dc-bab0-e2af943baadd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.612869 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.613345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.613549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.613806 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.614002 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.614111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] PATH_INFO: `/v3/projects/ef8044ead34847cbbc0401f752270fe6/users/487f6b4fac53420db417a67688d8f414/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.614366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.614646 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] PATH_INFO: `/v3/users/64c6297bbdb54bf6bfb5699abc97e0ed` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.615850 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] RBAC: Authorizing `identity:delete_user(user_id=64c6297bbdb54bf6bfb5699abc97e0ed)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.616107 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef8044ead34847cbbc0401f752270fe6, user_id=487f6b4fac53420db417a67688d8f414, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.617833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.618762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e3f0f-fe92-4f4d-87f7-6bb59f33cb39 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.621017 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:36.630077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1376/2782] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:36 2022] PUT /identity/v3/projects/ef8044ead34847cbbc0401f752270fe6/users/487f6b4fac53420db417a67688d8f414/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:36.640317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.640604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.640878 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.671009 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] MFA Rules not processed for user `487f6b4fac53420db417a67688d8f414`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:36.672178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:36.742648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1377/2783] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:36 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:36.753798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:36.755401 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:36.778282 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:36.819078 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c19ab8-5b43-4bff-8366-6156cf5486da None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:36.822860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:36.823500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:36.823776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:36.824045 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:36.835728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:36.837352 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:36.848294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1378/2784] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:36 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:36.873228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64c6297bbdb54bf6bfb5699abc97e0ed'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.971901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64c6297bbdb54bf6bfb5699abc97e0ed'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.972543 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64c6297bbdb54bf6bfb5699abc97e0ed'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.985349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64c6297bbdb54bf6bfb5699abc97e0ed'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.985980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64c6297bbdb54bf6bfb5699abc97e0ed'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:36.997778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1407/2785] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:36 2022] DELETE /identity/v3/users/64c6297bbdb54bf6bfb5699abc97e0ed => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:37.085759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:37.095744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d31e9-2e52-48f4-9ced-44c6aa441d90 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:37.189901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:37.189901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:37.189901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:37.189901 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:37.190465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:37.280488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:37.282144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:37.302000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1379/2786] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:37.688461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:37.689955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:37.766987 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c58094-693a-489e-8d16-edc5e0d4190a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:37.771668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:37.772431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:37.772713 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:37.772986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] PATH_INFO: `/v3/projects/e066418afba7455c88b091089225327f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:37.774156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e066418afba7455c88b091089225327f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:37.776121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:37.898066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e066418afba7455c88b091089225327f', 'request_id': 'req-52f163e3-224b-4c6a-87c7-118a208176b0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:37.904817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e066418afba7455c88b091089225327f', 'request_id': 'req-52f163e3-224b-4c6a-87c7-118a208176b0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.028586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e066418afba7455c88b091089225327f', 'request_id': 'req-52f163e3-224b-4c6a-87c7-118a208176b0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.050136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] The token cache is being invalidate because project e066418afba7455c88b091089225327f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:38.050136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.050136 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.050666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1408/2787] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:37 2022] DELETE /identity/v3/projects/e066418afba7455c88b091089225327f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:38.054749 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:38.067521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6424abe-8a77-44e1-b982-81e97bab75c2 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:38.125549 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:38.126168 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:38.126432 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:38.126704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] PATH_INFO: `/v3/users/71bccde638fa43e4bc2b199391502535` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:38.127862 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bccde638fa43e4bc2b199391502535)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:38.129800 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:38.146590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:38.340875 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bccde638fa43e4bc2b199391502535'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.342230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bccde638fa43e4bc2b199391502535'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.434176 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bccde638fa43e4bc2b199391502535'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.444268 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bccde638fa43e4bc2b199391502535'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.457250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d3ae4e0-e432-4adb-bb06-d9d97a3ee24a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bccde638fa43e4bc2b199391502535'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:38.463767 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1380/2788] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:38 2022] DELETE /identity/v3/users/71bccde638fa43e4bc2b199391502535 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:38.947594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:38.947594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:38.960682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f163e3-224b-4c6a-87c7-118a208176b0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:38.977785 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:39.028476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.029653 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.030218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.030218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] PATH_INFO: `/v3/projects/ed65b8455ea046edb50da1dd2a289509` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.031374 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed65b8455ea046edb50da1dd2a289509)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.037268 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.144477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed65b8455ea046edb50da1dd2a289509', 'request_id': 'req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:39.151339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed65b8455ea046edb50da1dd2a289509', 'request_id': 'req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:39.246064 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed65b8455ea046edb50da1dd2a289509', 'request_id': 'req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:39.258805 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] The token cache is being invalidate because project ed65b8455ea046edb50da1dd2a289509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:39.259081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:39.259784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4d66855d-3fe5-44d1-85a1-fe3e2e5cc42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:39.261587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1409/2789] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:38 2022] DELETE /identity/v3/projects/ed65b8455ea046edb50da1dd2a289509 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:39.266696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.267300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.267300 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.290961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:39.290961 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:39.350394 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1381/2790] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:39.417054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.417316 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.417587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.439519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:39.440735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:39.449124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1410/2791] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:39.454025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.464139 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ea459-695f-464a-acb3-b6a782af82dc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.469520 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.470167 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.470448 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.470725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.471428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:39.471731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.473400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.487081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1382/2792] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:39.491606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.501813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9fbae-ff00-4ebc-bc3e-e43da6e90b84 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.506754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.507387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.507662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.507930 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.508579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.510484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.665008 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1411/2793] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:39 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:39.669242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.684835 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257dbb8c-09d9-4729-b64b-a187339cd6fc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.745091 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.745707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.745987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.746260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.746898 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.749442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.840975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1383/2794] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:39 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:39.845828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.856338 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92d23f-c774-4c4d-8fd7-1851a3196784 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.862072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.862607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.862883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.863167 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.863816 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.865399 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.876679 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1412/2795] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:39.880821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.893420 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5014e-ee45-42e0-a84b-2135cc76e6ac admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.898214 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.898847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.899105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.899405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.900055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.901720 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375432b4-97d8-4e44-805d-11dd4c6e0adc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:39.908233 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:39.913474 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1384/2796] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:39.917792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:39.928278 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b66ec2-5bd7-40f1-9b0d-774ddff24458 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:39.933041 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:39.933695 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:39.933971 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:39.934243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] PATH_INFO: `/v3/projects/bb2a478df7974950996345ed92bd083f/users/0b92150e8ab24d0da0bf2d264a2892e6/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:39.950096 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2a478df7974950996345ed92bd083f, user_id=0b92150e8ab24d0da0bf2d264a2892e6, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:39.953371 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.029759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1413/2797] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:39 2022] PUT /identity/v3/projects/bb2a478df7974950996345ed92bd083f/users/0b92150e8ab24d0da0bf2d264a2892e6/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:40.041156 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.041594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.041979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.073572 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] MFA Rules not processed for user `0b92150e8ab24d0da0bf2d264a2892e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:40.074971 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:40.144316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1385/2798] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:40.148823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.193883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:40.221860 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b93a9-38d8-4d12-9311-679b093f92c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.227264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.228240 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.228786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.229020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.231122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.233413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.304436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1414/2799] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:40.308511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.320748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8af5d36-2b47-4d09-b3be-52c425d666d2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.384377 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.384892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.385164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.385430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.386081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.388129 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.458518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1386/2800] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:40.462624 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.473904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce49fa-6540-4bf4-9165-7543a1365fe9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.478840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.479482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.479759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.480027 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.480797 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.482728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.493648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1415/2801] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:40.498523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.510490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd264a61-1ade-4d3c-a947-b62b168918a5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.516019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.516620 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.516896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.517164 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.517807 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.519450 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48298bcd-99d5-4acb-934d-2ba8f966e0ca admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.529948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1387/2802] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:40.534106 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.544368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa1c6b-4ab1-478e-90ab-9729d83b71e3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.549586 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.550248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.550524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.550794 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] PATH_INFO: `/v3/projects/35a2c0a0040b430dbb18929f868b65cc/users/2e72ba23ba3b4338bce217d8c5cafe1f/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.566857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a2c0a0040b430dbb18929f868b65cc, user_id=2e72ba23ba3b4338bce217d8c5cafe1f, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.569556 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.647029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1416/2803] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:40 2022] PUT /identity/v3/projects/35a2c0a0040b430dbb18929f868b65cc/users/2e72ba23ba3b4338bce217d8c5cafe1f/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:40.665298 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.665688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.666153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.704809 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] MFA Rules not processed for user `2e72ba23ba3b4338bce217d8c5cafe1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:40.705517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:40.772590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:40.778058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1388/2804] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:40.783832 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.861640 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e0808-b8d1-44fc-82da-783fe53cd7fc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:40.866825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:40.867514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:40.867793 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:40.868061 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:40.868754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:40.870683 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:40.962189 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1417/2805] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:40.966190 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:40.977935 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912b529-4ca3-451a-a4e0-a6ce9adfff78 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.039508 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.040122 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.040397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.040663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.041366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.043413 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.583609 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1389/2806] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:40 2022] POST /identity/v3/users => generated 298 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:41.587403 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.598788 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941111fa-0900-41e1-850c-0cba6388d912 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.603477 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.604162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.604383 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.604656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.605366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.607532 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.620579 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1418/2807] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:41.624750 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.635269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240a406-b377-4e6f-981d-8982326dcfd1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.640454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.644163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.644163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.644163 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] PATH_INFO: `/v3/projects/7afb735baeca487ca53f18c076aed9d8/users/cdc09d5aa9864023a1b8c4817d480a26/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.657995 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afb735baeca487ca53f18c076aed9d8, user_id=cdc09d5aa9864023a1b8c4817d480a26, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.660829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.740397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1390/2808] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:41 2022] PUT /identity/v3/projects/7afb735baeca487ca53f18c076aed9d8/users/cdc09d5aa9864023a1b8c4817d480a26/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:41.748066 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.754742 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213a3a2-c971-4ff6-81a8-b3cb6e813cab admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.820944 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.821562 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.821849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.822128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.822808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.824446 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd5869-2159-4d7a-8a60-2654c0adecc1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.827497 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:41.838922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1419/2809] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:41.843342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:41.855478 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ef58a-339d-46c4-9a0a-f09b90db18e1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:41.864397 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:41.865194 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.871250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.871250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] PATH_INFO: `/v3/projects/7afb735baeca487ca53f18c076aed9d8/users/cdc09d5aa9864023a1b8c4817d480a26/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.871250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7afb735baeca487ca53f18c076aed9d8, user_id=cdc09d5aa9864023a1b8c4817d480a26, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:41.871250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:41.886930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1391/2810] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:41 2022] PUT /identity/v3/projects/7afb735baeca487ca53f18c076aed9d8/users/cdc09d5aa9864023a1b8c4817d480a26/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:41.900330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:41.900705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:41.901052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:41.934180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] MFA Rules not processed for user `cdc09d5aa9864023a1b8c4817d480a26`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:41.936223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:42.018430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1420/2811] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:42.030442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:42.032201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:42.113047 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578dbd28-2734-4242-ae85-d3c38864d8ca admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:42.123764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:42.123764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:42.123764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:42.123764 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:42.192364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:42.194077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fddc0b-92ca-4db0-8de4-abf3844deb6d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:42.203945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1392/2812] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:43.467796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:43.470181 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:43.539436 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac1c25-dd5d-4244-92ca-a6676f1d0eb0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:43.544791 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:43.545449 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:43.545724 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:43.545992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] PATH_INFO: `/v3/users/a71077ca7b1e46a1b472ea6d0c57f565` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:43.547293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71077ca7b1e46a1b472ea6d0c57f565)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:43.549252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:43.744998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71077ca7b1e46a1b472ea6d0c57f565'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:43.811849 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71077ca7b1e46a1b472ea6d0c57f565'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:43.812394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71077ca7b1e46a1b472ea6d0c57f565'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:43.825823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71077ca7b1e46a1b472ea6d0c57f565'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:43.826583 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71077ca7b1e46a1b472ea6d0c57f565'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:43.839124 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1421/2813] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:43 2022] DELETE /identity/v3/users/a71077ca7b1e46a1b472ea6d0c57f565 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:43.844543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:43.845130 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:43.845292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:43.853648 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:43.864994 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:43.866207 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:43.927521 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1393/2814] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:43.938897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:43.940550 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:43.946228 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:44.008740 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeefcc-bfa9-4b99-a613-05ba0ad0931c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:44.012833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:44.019137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:44.019137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:44.019137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:44.024203 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:44.025648 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:44.036730 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1422/2815] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:44.175986 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:44.177880 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:44.240224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b4ac-9d4e-4fb8-b4ba-79de189bf038 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:44.244383 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:44.254612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:44.254612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:44.254612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:44.254612 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:44.329039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:44.329039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:44.342575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1394/2816] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:44.639813 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:44.641587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:44.655420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904efa1-ed78-42d0-89bd-338049b14c2f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:44.660784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:44.661891 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:44.662221 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:44.662484 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] PATH_INFO: `/v3/projects/9d2be6f5420a4e8983c833ba4f3d6ee4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:44.663647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2be6f5420a4e8983c833ba4f3d6ee4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:44.665887 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:44.812122 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2be6f5420a4e8983c833ba4f3d6ee4', 'request_id': 'req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:44.820819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2be6f5420a4e8983c833ba4f3d6ee4', 'request_id': 'req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:44.902734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2be6f5420a4e8983c833ba4f3d6ee4', 'request_id': 'req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:44.919998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] The token cache is being invalidate because project 9d2be6f5420a4e8983c833ba4f3d6ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:44.925200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:44.925200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:44.925200 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1423/2817] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:44 2022] DELETE /identity/v3/projects/9d2be6f5420a4e8983c833ba4f3d6ee4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:44.927052 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:44.944146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec67ea4-f18c-41d7-a780-9c3232b3ac67 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:45.015371 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:45.016039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.016323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.016589 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] PATH_INFO: `/v3/users/487f6b4fac53420db417a67688d8f414` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.017782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=487f6b4fac53420db417a67688d8f414)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:45.019791 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:45.021816 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:45.302069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487f6b4fac53420db417a67688d8f414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.302944 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487f6b4fac53420db417a67688d8f414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.405090 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487f6b4fac53420db417a67688d8f414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.413751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487f6b4fac53420db417a67688d8f414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.428396 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487f6b4fac53420db417a67688d8f414'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:45.434659 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1395/2818] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:44 2022] DELETE /identity/v3/users/487f6b4fac53420db417a67688d8f414 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:45.882010 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:45.883547 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:45.897593 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db132a7d-8eea-4d69-b1bc-8a8e0af5d765 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:45.965751 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:45.966872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:45.967175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:45.967452 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] PATH_INFO: `/v3/projects/ef8044ead34847cbbc0401f752270fe6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:45.968642 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef8044ead34847cbbc0401f752270fe6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:45.970638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.052307 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:46.070013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef8044ead34847cbbc0401f752270fe6', 'request_id': 'req-7ca838ac-7f46-463f-b214-8e1d179cef78'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:46.082252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef8044ead34847cbbc0401f752270fe6', 'request_id': 'req-7ca838ac-7f46-463f-b214-8e1d179cef78'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:46.106903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.108435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:46.177518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe459da-0d3d-4353-b8a1-be121a3104a8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.201195 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef8044ead34847cbbc0401f752270fe6', 'request_id': 'req-7ca838ac-7f46-463f-b214-8e1d179cef78'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:46.213885 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] The token cache is being invalidate because project ef8044ead34847cbbc0401f752270fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:46.214198 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:46.214857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7ca838ac-7f46-463f-b214-8e1d179cef78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:46.217563 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1424/2819] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:45 2022] DELETE /identity/v3/projects/ef8044ead34847cbbc0401f752270fe6 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:46.222114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1d87-8f5c-459e-abbd-6d4b83d38440 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.222397 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1d87-8f5c-459e-abbd-6d4b83d38440 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.222655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd1d87-8f5c-459e-abbd-6d4b83d38440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.234957 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.235630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.235929 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.236183 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.238087 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:46.244019 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-babd1d87-8f5c-459e-abbd-6d4b83d38440 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:46.248746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-babd1d87-8f5c-459e-abbd-6d4b83d38440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:46.306538 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:46.312464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1425/2820] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:46.321164 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.321453 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.321755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.330641 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.332336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.343054 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:46.344324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:46.345392 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1396/2821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:46.353312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1426/2822] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:46.358545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.369416 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0f72b-23a3-4fd8-b15b-30fda1252a28 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.374772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.375551 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.375897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.376320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.377414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:46.377526 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.379547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.395847 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1397/2823] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:46:46.396977 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.407113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f2ae1-e838-49c4-b96e-191e945ba75b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.412133 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.412734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.413012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.413280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.413924 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.415863 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.533310 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1427/2824] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:46 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:46.538471 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.548762 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c08132-994c-44a9-ac6d-316480dda6a3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.615636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.616249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.616532 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.616810 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.617458 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.619437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.771550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1398/2825] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:46 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:46.776086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.788706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8a530-15ca-436a-9fd0-77e9e7ccf62a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.792729 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.793540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.794339 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.794617 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.795304 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.796920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.808320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1428/2826] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:46.813461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.824039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33173ebe-d6da-4320-8bac-8a470e6977e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.829200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.829813 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.830155 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.830427 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.831143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.833118 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d848c6-3fca-4c51-858b-4c77f995fb7b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.845023 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1399/2827] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:46.849187 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:46.862471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d260f6d-914b-466a-bdb3-4c3135256a56 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:46.865985 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:46.866670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:46.866947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:46.867257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] PATH_INFO: `/v3/projects/396cc56d10564e31bf09d3ad57e45947/users/d5571e2ec2f342978087268582772b1e/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:46.883904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=396cc56d10564e31bf09d3ad57e45947, user_id=d5571e2ec2f342978087268582772b1e, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:46.886557 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:46.932324 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1429/2828] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:46 2022] PUT /identity/v3/projects/396cc56d10564e31bf09d3ad57e45947/users/d5571e2ec2f342978087268582772b1e/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:47.004579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.005074 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.005355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.035721 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] MFA Rules not processed for user `d5571e2ec2f342978087268582772b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:47.036952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:47.107928 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1400/2829] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:47.112539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.172627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:47.178447 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19f467-a304-4ec3-9f53-3894db6a4eeb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.183544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.184158 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.184491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.184889 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.185546 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.188087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.301947 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1430/2830] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/projects => generated 383 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:46:47.303927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.320336 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f43fb-7641-48a3-9a90-f22426665a23 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.365830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:47.389409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.390029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.390306 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.390602 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.391288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.393405 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.525363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1401/2831] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/users => generated 298 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:47.529837 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.541775 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d71c-0759-4a14-8223-fba3fa2879d8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.547475 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.548089 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.548502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.548773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.549416 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.551020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.563590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1431/2832] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:47.567915 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.578320 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175eac6a-5da2-471b-80f0-82246673fbb5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.584801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.584801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.584801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.584801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] PATH_INFO: `/v3/projects/4f870475da8f4daa919650bbed53ba87/users/d067c3e58cc9406dbf62e31b9fcfea22/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.609494 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f870475da8f4daa919650bbed53ba87, user_id=d067c3e58cc9406dbf62e31b9fcfea22, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.614279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.749063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1402/2833] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:47 2022] PUT /identity/v3/projects/4f870475da8f4daa919650bbed53ba87/users/d067c3e58cc9406dbf62e31b9fcfea22/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:47.753249 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.763637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78638c01-613c-4092-bfc6-1052fa9bb9b5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.823647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.824260 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.824539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.824810 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.825565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.827353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f01e60-090c-427e-afe0-df87ffb2555e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.837706 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1432/2834] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:47.842896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:47.853794 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ac131-cbed-45ed-ab94-f20d6f5e542f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:47.859305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:47.859980 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.860260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.860533 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] PATH_INFO: `/v3/projects/4f870475da8f4daa919650bbed53ba87/users/d067c3e58cc9406dbf62e31b9fcfea22/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.862501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f870475da8f4daa919650bbed53ba87, user_id=d067c3e58cc9406dbf62e31b9fcfea22, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:47.865195 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:47.876784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1403/2835] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:47 2022] PUT /identity/v3/projects/4f870475da8f4daa919650bbed53ba87/users/d067c3e58cc9406dbf62e31b9fcfea22/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:47.893425 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:47.893916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:47.894031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:47.925377 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] MFA Rules not processed for user `d067c3e58cc9406dbf62e31b9fcfea22`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:47.926628 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:48.006246 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1433/2836] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:48.018097 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:48.019996 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:48.111437 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9092da62-8d01-409f-aa4f-7c17b9fed3db admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:48.116115 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:48.116755 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:48.117030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:48.117294 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:48.127031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:48.128433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:48.138557 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1404/2837] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:48.289838 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:48.291380 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:48.303896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f8c44-a128-4b72-bff9-cb6ebfd9592a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:48.307670 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:48.308276 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:48.308551 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:48.308859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:48.362072 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:48.372445 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:48.374415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b435cc-69d0-4091-8ef3-84f60856774b service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:48.391274 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1434/2838] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:51.147836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:51.149322 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:51.224518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb6af2-d4ea-4109-8f0e-4bf518075472 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:51.229279 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:51.229922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:51.230200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:51.230473 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] PATH_INFO: `/v3/users/0b92150e8ab24d0da0bf2d264a2892e6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:51.234024 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b92150e8ab24d0da0bf2d264a2892e6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:51.235979 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:51.379512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:51.389305 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b92150e8ab24d0da0bf2d264a2892e6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:51.390008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b92150e8ab24d0da0bf2d264a2892e6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:51.485829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b92150e8ab24d0da0bf2d264a2892e6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:51.495649 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b92150e8ab24d0da0bf2d264a2892e6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:51.509507 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b92150e8ab24d0da0bf2d264a2892e6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:51.516560 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1405/2839] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:51 2022] DELETE /identity/v3/users/0b92150e8ab24d0da0bf2d264a2892e6 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:51.522332 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:51.522823 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:51.522928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:51.543069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:51.544502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:51.609104 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1435/2840] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:51.618725 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:51.620175 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:51.681626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e51b-395d-48ca-a6ed-84380bae4bdd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:51.686347 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:51.686958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:51.687265 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:51.695209 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:51.697171 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:51.698626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:51.709771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1406/2841] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:52.211940 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:52.213408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:52.227179 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589334b-56d9-4dfc-8f10-16ad95d705ee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:52.233571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:52.237877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:52.237877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:52.237877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] PATH_INFO: `/v3/projects/bb2a478df7974950996345ed92bd083f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:52.237877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2a478df7974950996345ed92bd083f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:52.237877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:52.401100 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2a478df7974950996345ed92bd083f', 'request_id': 'req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.408972 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2a478df7974950996345ed92bd083f', 'request_id': 'req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.514634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2a478df7974950996345ed92bd083f', 'request_id': 'req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.529603 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] The token cache is being invalidate because project bb2a478df7974950996345ed92bd083f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:52.530117 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.530753 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.535575 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1436/2842] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:52 2022] DELETE /identity/v3/projects/bb2a478df7974950996345ed92bd083f => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:52.536812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:52.550127 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b13f8-36af-4aaa-a0b7-a5ee9079363e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:52.566134 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:52.568599 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:52.629587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:52.630357 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:52.630525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:52.630801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] PATH_INFO: `/v3/users/2e72ba23ba3b4338bce217d8c5cafe1f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:52.632012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e72ba23ba3b4338bce217d8c5cafe1f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:52.634368 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:52.669411 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c8b69d-7d7d-4cea-9eca-3e5fcac963fb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:52.675360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:52.676046 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:52.676329 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:52.676606 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] PATH_INFO: `/v3/users/d5571e2ec2f342978087268582772b1e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:52.677776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5571e2ec2f342978087268582772b1e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:52.679756 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:52.696560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:52.856564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e72ba23ba3b4338bce217d8c5cafe1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.857226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e72ba23ba3b4338bce217d8c5cafe1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.959060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e72ba23ba3b4338bce217d8c5cafe1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.965826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:52.969545 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e72ba23ba3b4338bce217d8c5cafe1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.977430 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5571e2ec2f342978087268582772b1e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.985590 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e72ba23ba3b4338bce217d8c5cafe1f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:52.992292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1407/2843] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:52 2022] DELETE /identity/v3/users/2e72ba23ba3b4338bce217d8c5cafe1f => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:53.168825 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5571e2ec2f342978087268582772b1e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.169694 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5571e2ec2f342978087268582772b1e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.182511 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.184206 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.188265 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5571e2ec2f342978087268582772b1e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.188673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e890f96d-de33-4d7b-9a8d-f532e5925505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5571e2ec2f342978087268582772b1e'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.204082 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1437/2844] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:52 2022] DELETE /identity/v3/users/d5571e2ec2f342978087268582772b1e => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:53.208647 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.209075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.209658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.237926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:53.239382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.294121 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815801-5b31-4d84-b14d-bd266b19433a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.317489 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1438/2845] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:53.327960 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.329781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.372055 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.372608 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.372893 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.373170 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.375199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.388310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.390706 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:53.400467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d94e0-7c4b-4ee5-87e2-4bca875248aa None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.404365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1408/2846] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:53.405123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.405754 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.406035 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.406308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.416922 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.419029 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:53.430026 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1439/2847] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:53.453509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.456027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.472403 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98780ed5-6106-4863-8030-c03474e987d4 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.481304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.486573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.486573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.486573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] PATH_INFO: `/v3/projects/35a2c0a0040b430dbb18929f868b65cc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.486573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a2c0a0040b430dbb18929f868b65cc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.486573 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:53.628627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a2c0a0040b430dbb18929f868b65cc', 'request_id': 'req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.635177 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a2c0a0040b430dbb18929f868b65cc', 'request_id': 'req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.719039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a2c0a0040b430dbb18929f868b65cc', 'request_id': 'req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.719039 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:53.726734 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] The token cache is being invalidate because project 35a2c0a0040b430dbb18929f868b65cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:53.727344 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.728153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:53.729988 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1409/2848] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:53 2022] DELETE /identity/v3/projects/35a2c0a0040b430dbb18929f868b65cc => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:53.734485 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.744807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e473300-bd94-45f2-b089-90f65f37333f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.790614 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:53.817239 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.817872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.818153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.818422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] PATH_INFO: `/v3/users/cdc09d5aa9864023a1b8c4817d480a26` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.819620 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc09d5aa9864023a1b8c4817d480a26)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.821627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:53.917486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:53.917486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:53.927513 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dae34-2f55-4b20-a6f8-d6dd900e2738 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:53.932561 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:53.933212 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:53.933495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:53.934941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] PATH_INFO: `/v3/projects/396cc56d10564e31bf09d3ad57e45947` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:53.939795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=396cc56d10564e31bf09d3ad57e45947)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:53.941286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:54.089432 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc09d5aa9864023a1b8c4817d480a26'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.095782 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396cc56d10564e31bf09d3ad57e45947', 'request_id': 'req-c422ef8c-c802-4299-ad16-bd5b381ac063'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.102552 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396cc56d10564e31bf09d3ad57e45947', 'request_id': 'req-c422ef8c-c802-4299-ad16-bd5b381ac063'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.187644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc09d5aa9864023a1b8c4817d480a26'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.188292 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc09d5aa9864023a1b8c4817d480a26'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.202534 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc09d5aa9864023a1b8c4817d480a26'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.203801 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc09d5aa9864023a1b8c4817d480a26'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.217267 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1440/2849] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:53 2022] DELETE /identity/v3/users/cdc09d5aa9864023a1b8c4817d480a26 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:54.250674 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396cc56d10564e31bf09d3ad57e45947', 'request_id': 'req-c422ef8c-c802-4299-ad16-bd5b381ac063'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.264292 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] The token cache is being invalidate because project 396cc56d10564e31bf09d3ad57e45947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:54.264870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.265811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.267726 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1410/2850] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:53 2022] DELETE /identity/v3/projects/396cc56d10564e31bf09d3ad57e45947 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:54.271728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:54.273429 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:54.348601 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349523be-9c75-4573-9191-55f1cde5ec50 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:54.353372 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:54.354105 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:54.354384 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:54.354658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] PATH_INFO: `/v3/users/d067c3e58cc9406dbf62e31b9fcfea22` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:54.355878 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d067c3e58cc9406dbf62e31b9fcfea22)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:54.357853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:54.594209 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd067c3e58cc9406dbf62e31b9fcfea22'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.686355 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:54.688682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:54.710472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd067c3e58cc9406dbf62e31b9fcfea22'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.710472 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd067c3e58cc9406dbf62e31b9fcfea22'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.711387 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:54.726296 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd067c3e58cc9406dbf62e31b9fcfea22'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.728287 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9ef6acea-c8ef-482d-ac0a-cc1cb7cfd96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd067c3e58cc9406dbf62e31b9fcfea22'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:54.742376 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1441/2851] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:54 2022] DELETE /identity/v3/users/d067c3e58cc9406dbf62e31b9fcfea22 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:46:54.771696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422ef8c-c802-4299-ad16-bd5b381ac063 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:54.853777 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:54.854426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:54.854704 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:54.854975 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] PATH_INFO: `/v3/projects/7afb735baeca487ca53f18c076aed9d8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:54.856364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7afb735baeca487ca53f18c076aed9d8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:54.858482 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:54.884329 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:55.041200 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7afb735baeca487ca53f18c076aed9d8', 'request_id': 'req-e238ad1a-0725-4e80-9f9d-56b3951f555a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.049630 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7afb735baeca487ca53f18c076aed9d8', 'request_id': 'req-e238ad1a-0725-4e80-9f9d-56b3951f555a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.170417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7afb735baeca487ca53f18c076aed9d8', 'request_id': 'req-e238ad1a-0725-4e80-9f9d-56b3951f555a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.182391 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] The token cache is being invalidate because project 7afb735baeca487ca53f18c076aed9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:55.182682 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.183390 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.186041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1411/2852] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:54 2022] DELETE /identity/v3/projects/7afb735baeca487ca53f18c076aed9d8 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:55.190773 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43ed-5840-4d3e-b4b5-1d684b0f9113 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.191057 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43ed-5840-4d3e-b4b5-1d684b0f9113 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.191398 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec43ed-5840-4d3e-b4b5-1d684b0f9113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.214346 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5cec43ed-5840-4d3e-b4b5-1d684b0f9113 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:55.216175 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5cec43ed-5840-4d3e-b4b5-1d684b0f9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.291134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.291134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.291427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1442/2853] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:55.292588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.292877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.293148 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.312398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238ad1a-0725-4e80-9f9d-56b3951f555a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.321692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:55.321692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.321892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.321892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.321892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.321892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] PATH_INFO: `/v3/projects/4f870475da8f4daa919650bbed53ba87` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.322514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f870475da8f4daa919650bbed53ba87)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.325270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.328073 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1443/2854] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:55.332168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.344778 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0f231-7ec6-4474-94ac-1e47099cea72 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.352162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.352162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.352162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.352162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.352619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:55.352619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.352619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.369464 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1444/2855] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:46:55.374458 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.388285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74848708-53ba-4d01-9c31-07982282a243 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.391712 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.392357 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.392634 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.392901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.393566 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.405345 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.474739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f870475da8f4daa919650bbed53ba87', 'request_id': 'req-02d29969-48e0-4d0a-ba0a-f439f6eb1358'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.481073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f870475da8f4daa919650bbed53ba87', 'request_id': 'req-02d29969-48e0-4d0a-ba0a-f439f6eb1358'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.521644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1445/2856] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:55.525802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.595781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e0454-9105-4627-b579-ab0c497b0902 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.602693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.603716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.604137 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.604467 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.605156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.608110 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.617917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f870475da8f4daa919650bbed53ba87', 'request_id': 'req-02d29969-48e0-4d0a-ba0a-f439f6eb1358'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.632337 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] The token cache is being invalidate because project 4f870475da8f4daa919650bbed53ba87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:55.632655 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.633330 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-02d29969-48e0-4d0a-ba0a-f439f6eb1358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:55.634865 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1412/2857] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:55 2022] DELETE /identity/v3/projects/4f870475da8f4daa919650bbed53ba87 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:55.640199 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32dd0c-e1d0-4cf8-9afe-3575035e5420 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.640493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32dd0c-e1d0-4cf8-9afe-3575035e5420 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.640768 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca32dd0c-e1d0-4cf8-9afe-3575035e5420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.660805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ca32dd0c-e1d0-4cf8-9afe-3575035e5420 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:55.662029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca32dd0c-e1d0-4cf8-9afe-3575035e5420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.670840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1413/2858] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:55.680013 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.680293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.680594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.701838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:55.703062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.711834 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1414/2859] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:55.716587 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1446/2860] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:55.716731 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.719320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.721350 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.726672 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84d775-fa5c-4471-960c-beba4775e815 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.731534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.732031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.732317 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.732582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.733286 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:46:55.733548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.734156 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd34a5e-f909-468f-bc5f-db98d9849841 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.735182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.738903 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.740131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.740131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.740362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.740901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.742505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.747617 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1415/2861] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 01 16:46:55.751972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.756554 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1447/2862] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:55.757796 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.763104 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe897a61-0a84-43c8-a35c-94b399b088ca admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.776018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.776018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.776018 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.776434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76645d6-6109-42f6-aa16-ff697d36b598 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.776550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.776550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.776550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.776819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.776819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.776819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.776819 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.777153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.777360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.784471 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:55.790561 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1448/2863] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:55.794828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.805033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929f2e2-93ea-4c8c-a98f-7bba701eb020 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:55.809817 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:55.810555 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.811225 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.811569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] PATH_INFO: `/v3/projects/5d2256a9d220490f832cdbc3919c1839/users/3c199a62af0244ff9aeeb8a7c61332fe/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.829243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d2256a9d220490f832cdbc3919c1839, user_id=3c199a62af0244ff9aeeb8a7c61332fe, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:55.832016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3646dcc-cefd-4cb0-ab9d-55efcb6c52a9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:55.887821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1416/2864] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:55.891900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:55.898904 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1449/2865] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:55 2022] PUT /identity/v3/projects/5d2256a9d220490f832cdbc3919c1839/users/3c199a62af0244ff9aeeb8a7c61332fe/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:55.918763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:55.919075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:55.919375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:55.948043 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] MFA Rules not processed for user `3c199a62af0244ff9aeeb8a7c61332fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:55.948442 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:55.949862 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:55.963518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f7339-21ee-4a67-b783-a9561fc45a9f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.027058 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.027839 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.028189 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.028534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.029251 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.031436 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.043610 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1450/2866] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:56.052639 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.054809 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:56.119085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced013a-c7be-4405-9b87-d9b7170b0a21 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.123673 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.124289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.124569 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.124845 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.137426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1417/2867] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:46:55 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:46:56.137676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.137676 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.139066 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.146455 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1451/2868] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:56.153647 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21067ce5-2ee0-4cb2-98a8-fc1742087ff7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.158613 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.159273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.159541 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.159811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.160460 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.162076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.172532 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.173031 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1418/2869] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:46:56.176845 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.187627 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772eada2-6fed-4ef7-84c0-06c0d47e812e service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.187899 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ccbe0-686c-4604-a148-9014de7c7612 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.192509 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.192662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.193145 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.193288 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.193428 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.193582 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.193699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] PATH_INFO: `/v3/users/3c199a62af0244ff9aeeb8a7c61332fe` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.193829 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.194370 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.195094 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c199a62af0244ff9aeeb8a7c61332fe)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.196069 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.197328 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.208241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1419/2870] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 01 16:46:56.212575 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.222784 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbffb38-97cc-4ac8-937e-5695d779cb38 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.227605 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.228267 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.228547 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.228820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] PATH_INFO: `/v3/projects/9f944cd7895d4cc4befa60ea9074219a/users/591e9448bffb4ae0bfceb7091e89b3be/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.249398 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f944cd7895d4cc4befa60ea9074219a, user_id=591e9448bffb4ae0bfceb7091e89b3be, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.250186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ab85f-8870-4e96-8c8b-79b8de0152f8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.381946 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1420/2871] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:46:56 2022] PUT /identity/v3/projects/9f944cd7895d4cc4befa60ea9074219a/users/591e9448bffb4ae0bfceb7091e89b3be/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:56.392709 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.392989 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.393263 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.423626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] MFA Rules not processed for user `591e9448bffb4ae0bfceb7091e89b3be`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:56.424860 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:56.507236 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1421/2872] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:46:56.507637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c199a62af0244ff9aeeb8a7c61332fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:56.518626 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.548671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:56.574982 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35dd53-568b-47b6-85cb-6ecd9266d232 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.576651 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c199a62af0244ff9aeeb8a7c61332fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:56.577360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c199a62af0244ff9aeeb8a7c61332fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:56.578832 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.579475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.579746 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.580025 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.586020 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:46:56.589820 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.592126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c247480-65ff-41ef-abf2-fbd01f6df78a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.592726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c199a62af0244ff9aeeb8a7c61332fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:56.593520 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c199a62af0244ff9aeeb8a7c61332fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:56.602043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1422/2873] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:46:56.608114 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1452/2874] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:46:56 2022] DELETE /identity/v3/users/3c199a62af0244ff9aeeb8a7c61332fe => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 01 16:46:56.612743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.613178 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.613284 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.633515 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:46:56.634729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:56.702365 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1423/2875] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:46:56.712415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:56.713920 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:56.780393 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fe0-41c7-44af-959f-6a4e2b42e2e0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:56.784826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:56.785488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:56.785766 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:56.786038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:56.795804 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:56.797248 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70879a-5de1-4708-bb45-a95d4cc738db service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:56.807799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1453/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:46:57.226619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:46:57.228984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:46:57.255372 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131ca45-5fcf-4cd6-a2b4-0b5acdc94906 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:46:57.273795 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:46:57.274964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:46:57.275488 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:46:57.275964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] PATH_INFO: `/v3/projects/5d2256a9d220490f832cdbc3919c1839` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:46:57.277500 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d2256a9d220490f832cdbc3919c1839)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:46:57.280373 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:46:57.458934 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d2256a9d220490f832cdbc3919c1839', 'request_id': 'req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:57.465422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d2256a9d220490f832cdbc3919c1839', 'request_id': 'req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:57.552841 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d2256a9d220490f832cdbc3919c1839', 'request_id': 'req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:57.564256 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] The token cache is being invalidate because project 5d2256a9d220490f832cdbc3919c1839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:46:57.564550 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:57.567064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:46:57.567064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1424/2877] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:46:57 2022] DELETE /identity/v3/projects/5d2256a9d220490f832cdbc3919c1839 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:02.143420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:02.144180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:02.144180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:02.164916 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:02.166152 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:02.229131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1454/2878] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Sep 1 16:47:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:47:02.357454 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:02.358857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:02.414639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:02.439514 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b9617-a4e6-4fae-8293-eb5bc34b3d2e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:02.444770 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:02.445393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:02.445669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:02.445941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:02.455287 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:02.456918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96412a30-bc62-46bf-8641-192665a54ad5 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:02.466509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1425/2879] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:47:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:47:12.221859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=59,keystone:DELETE=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:12.500927 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:12.850003 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:12.851980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:12.959330 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eda2fbb-6a57-4cff-9b85-267ea6e8f9ad None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:12.964888 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:12.966055 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:12.966461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:12.966844 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] PATH_INFO: `/v3/users/591e9448bffb4ae0bfceb7091e89b3be` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:12.968184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591e9448bffb4ae0bfceb7091e89b3be)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:12.970354 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:13.218763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591e9448bffb4ae0bfceb7091e89b3be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:13.344006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591e9448bffb4ae0bfceb7091e89b3be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:13.346111 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591e9448bffb4ae0bfceb7091e89b3be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:13.366389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591e9448bffb4ae0bfceb7091e89b3be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:13.367859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591e9448bffb4ae0bfceb7091e89b3be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:13.392074 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1455/2880] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:47:12 2022] DELETE /identity/v3/users/591e9448bffb4ae0bfceb7091e89b3be => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:13.399004 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:13.399501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:13.399959 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:13.435801 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:13.437616 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:13.539424 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1426/2881] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:13.549401 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:13.550956 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:13.638865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42828d3-440e-439a-a2ef-c36738737b03 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:13.642824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:13.643567 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:13.643859 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:13.644151 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:13.670803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:13.672504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f925a-fb38-4b17-863f-d209050ed030 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:13.689576 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1456/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:14.280128 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:14.281825 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:14.295179 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765b868-2cbc-4b18-94dc-915771184559 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:14.300423 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:14.301296 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:14.301579 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:14.301852 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] PATH_INFO: `/v3/projects/9f944cd7895d4cc4befa60ea9074219a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:14.303054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f944cd7895d4cc4befa60ea9074219a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:14.305146 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:14.758728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f944cd7895d4cc4befa60ea9074219a', 'request_id': 'req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:14.768363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f944cd7895d4cc4befa60ea9074219a', 'request_id': 'req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:14.877729 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f944cd7895d4cc4befa60ea9074219a', 'request_id': 'req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:14.896138 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] The token cache is being invalidate because project 9f944cd7895d4cc4befa60ea9074219a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:47:14.897798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:14.897798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2ca8962e-7514-4e8b-8f93-76ccd9248e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:14.898863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1427/2883] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:47:14 2022] DELETE /identity/v3/projects/9f944cd7895d4cc4befa60ea9074219a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:14.905847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:14.906741 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:14.907218 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:14.939482 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:14.941087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:14.968662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:15.031308 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1457/2884] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:15.035030 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.035453 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.035856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.058854 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:15.060911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:15.070486 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1428/2885] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:47:15.076300 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.087461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb3621-7c46-4d03-a5f9-7d5568d82498 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.100405 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.101188 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.101573 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.101946 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.102722 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:47:15.103129 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.104943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.118926 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1458/2886] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:47:15.123836 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.135765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc7bf3-215a-462c-9527-27dc44f2beba None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.141882 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.142625 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.143053 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.143400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.144063 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.146102 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.240186 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1429/2887] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:47:15 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:47:15.244840 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.256413 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869e4fe-a554-481e-b9f0-fe9407334e67 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.325644 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.326476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.326821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.327373 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.328119 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.330168 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.451623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1459/2888] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:47:15 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:47:15.455895 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.473660 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dbd03-fa69-4e99-8178-e513878ed6eb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.480992 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.482012 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.482519 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.483062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.483945 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.485772 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.501578 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1430/2889] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:15.506153 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.519847 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2ac3a-0cfc-40b7-958d-ebe06f42ea3a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.524757 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.525360 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.525638 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.526036 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.526687 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.528767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de35d5f-2b48-40a8-8087-117e96cdb2a7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.542336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1460/2890] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:15.546828 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.558220 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d5a67-3451-4aca-9d89-7be62ce02667 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.565518 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.566409 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.566691 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.566968 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] PATH_INFO: `/v3/projects/f3a0a22cbf574134969550017b976494/users/9411a8e305cc480ea5f85549e031e0f2/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.588905 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a0a22cbf574134969550017b976494, user_id=9411a8e305cc480ea5f85549e031e0f2, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:15.591881 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:15.695112 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1431/2891] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:47:15 2022] PUT /identity/v3/projects/f3a0a22cbf574134969550017b976494/users/9411a8e305cc480ea5f85549e031e0f2/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:15.706905 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957406a-68cf-4c0e-b10e-8373b3154f7f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.707319 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957406a-68cf-4c0e-b10e-8373b3154f7f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.707637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957406a-68cf-4c0e-b10e-8373b3154f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:15.741728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4957406a-68cf-4c0e-b10e-8373b3154f7f None None] MFA Rules not processed for user `9411a8e305cc480ea5f85549e031e0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:15.743143 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4957406a-68cf-4c0e-b10e-8373b3154f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:15.850015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1461/2892] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:15.858786 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:15.861346 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:15.890489 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:15.978491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3ea2a-1ff3-4ad0-8fe2-deaacbe65f0f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:15.986316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:15.986948 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:15.987250 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:15.987523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:16.010124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:16.015021 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:16.033669 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1432/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:20.281323 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:20.281834 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:20.281990 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:20.301733 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] MFA Rules not processed for user `711f1b63cfa1461da6b5b9bb286c6f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:20.302943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:20.318942 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:20.367746 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1462/2894] 10.222.0.16 () {58 vars in 967 bytes} [Thu Sep 1 16:47:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:20.380619 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:20.382060 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:20.455140 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824b110-a1f0-44de-815e-4b36626bc57a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:20.460748 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd247a8be3add4804b86dcc0cd4cb72e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:20.461362 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:20.461665 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:20.461933 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:20.470491 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:20.472111 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:20.482407 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1433/2895] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Sep 1 16:47:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 01 16:47:23.884608 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:23.886943 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:23.973281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82017f2-1a0b-4057-874f-8537aecfa01f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:23.978272 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:23.978912 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:23.979214 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:23.979504 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] PATH_INFO: `/v3/users/a2c7edbbe80e477ca0bfcd3de5fa27e9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:23.982382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c7edbbe80e477ca0bfcd3de5fa27e9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:23.983803 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:24.217898 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c7edbbe80e477ca0bfcd3de5fa27e9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:24.312053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c7edbbe80e477ca0bfcd3de5fa27e9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:24.312675 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c7edbbe80e477ca0bfcd3de5fa27e9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:24.327979 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c7edbbe80e477ca0bfcd3de5fa27e9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:24.328727 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7a47e4c0-20f6-4e63-9363-c96e55cf85ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c7edbbe80e477ca0bfcd3de5fa27e9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:24.341623 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1463/2896] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:47:23 2022] DELETE /identity/v3/users/a2c7edbbe80e477ca0bfcd3de5fa27e9 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:24.861269 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:24.873048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f8ed4-5b04-4cce-ba77-ee3ca5bdac05 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:24.942182 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:24.942919 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:24.943333 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:24.943723 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] PATH_INFO: `/v3/projects/149dd19b0cf147e7b26ad558b74cf85c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:24.944998 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149dd19b0cf147e7b26ad558b74cf85c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:24.947068 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.078913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149dd19b0cf147e7b26ad558b74cf85c', 'request_id': 'req-af013446-92a0-48f1-bda7-2c919491be61'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:25.085428 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149dd19b0cf147e7b26ad558b74cf85c', 'request_id': 'req-af013446-92a0-48f1-bda7-2c919491be61'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:25.091393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:25.202350 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149dd19b0cf147e7b26ad558b74cf85c', 'request_id': 'req-af013446-92a0-48f1-bda7-2c919491be61'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:25.214900 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] The token cache is being invalidate because project 149dd19b0cf147e7b26ad558b74cf85c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:47:25.215343 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:25.216080 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-af013446-92a0-48f1-bda7-2c919491be61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:25.217759 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1434/2897] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:47:24 2022] DELETE /identity/v3/projects/149dd19b0cf147e7b26ad558b74cf85c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:25.223244 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.224391 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.224846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.248955 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:25.250658 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:25.298784 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:25.322140 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1464/2898] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:25.330493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.330803 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.331048 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.353204 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:25.354483 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:25.364430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1435/2899] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:25.368995 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.380023 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de120af2-e924-47fc-ae1a-029b1f6b3aa7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.386667 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.387326 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.387607 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.387877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.388574 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 01 16:47:25.388896 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.390693 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.405993 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1465/2900] 10.222.0.16 () {58 vars in 1153 bytes} [Thu Sep 1 16:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 01 16:47:25.407219 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.418693 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdddb349-9ad8-461f-ab7e-c1cf0c8f1df8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.423941 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.424627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.424903 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.425173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.425812 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.427788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.528180 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1436/2901] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:47:25 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:47:25.529335 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.539745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e807c04-8cb8-4e8d-841a-a2067765d74e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.601505 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.602121 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.602395 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.602665 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.603518 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.606349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.728191 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1466/2902] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:47:25 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 01 16:47:25.732930 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.743323 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448ce3d-f6cb-4c12-959f-50356a828b7d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.749153 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.749758 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.750032 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.750301 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.750947 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.752600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.764321 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1437/2903] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:25.773988 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.785698 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedef74-163b-469b-8d36-62e2d2144659 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.790680 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.791358 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.791633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.791897 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.792643 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.794207 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da73f68-7631-4236-bc07-55646fc8d827 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.805517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1467/2904] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:25.811512 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:25.821639 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87166b-895c-40a3-b138-fecd5da61863 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:25.827119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:25.827799 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:25.828076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:25.828348 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] PATH_INFO: `/v3/projects/38f1672c9b5147fe8fef8659e7953fef/users/ec2aa8d7d4474e5b802c91c4cb5958cd/roles/184b9cf9fc5144009f8b26213f237b6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:25.844627 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38f1672c9b5147fe8fef8659e7953fef, user_id=ec2aa8d7d4474e5b802c91c4cb5958cd, role_id=184b9cf9fc5144009f8b26213f237b6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:25.847366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:25.997439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1438/2905] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:47:25 2022] PUT /identity/v3/projects/38f1672c9b5147fe8fef8659e7953fef/users/ec2aa8d7d4474e5b802c91c4cb5958cd/roles/184b9cf9fc5144009f8b26213f237b6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:26.002389 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41896559-b27f-4036-b159-06e4f62002ce None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:26.002734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41896559-b27f-4036-b159-06e4f62002ce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:26.003004 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41896559-b27f-4036-b159-06e4f62002ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:26.025215 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-41896559-b27f-4036-b159-06e4f62002ce None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:26.027006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-41896559-b27f-4036-b159-06e4f62002ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:26.088588 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1468/2906] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 01 16:47:26.099244 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:26.101043 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:26.161174 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e5b04-5480-4baa-b72a-eae603d1e528 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:26.164950 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:26.165559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:26.165897 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:26.166173 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:26.176447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:26.177911 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc8641d-aa12-4b6b-a3cd-9b0669873e23 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:26.188696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1439/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:29.662252 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41896559-b27f-4036-b159-06e4f62002ce None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:29.663710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-41896559-b27f-4036-b159-06e4f62002ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:29.676656 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41896559-b27f-4036-b159-06e4f62002ce None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:29.682099 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:29.683419 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:29.683710 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:29.683983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] PATH_INFO: `/v3/users/9411a8e305cc480ea5f85549e031e0f2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:29.685692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9411a8e305cc480ea5f85549e031e0f2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:29.687910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:29.952289 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:29.956526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9411a8e305cc480ea5f85549e031e0f2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:30.025469 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9411a8e305cc480ea5f85549e031e0f2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:30.026275 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9411a8e305cc480ea5f85549e031e0f2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:30.041353 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9411a8e305cc480ea5f85549e031e0f2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:30.042441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9411a8e305cc480ea5f85549e031e0f2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:30.062241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1469/2908] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:47:29 2022] DELETE /identity/v3/users/9411a8e305cc480ea5f85549e031e0f2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:30.063892 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:30.064406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:30.064739 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:30.087544 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] MFA Rules not processed for user `f9fb5099a6c24abebce19b1a9dfe02b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:30.088837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:30.156690 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1440/2909] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:30.170120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:30.172590 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:30.242848 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed843-46be-425d-a91b-9d1a70e468f5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:30.250365 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:30.251085 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:30.251412 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:30.251685 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:30.268877 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:30.270767 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787e0af-333c-4953-9ae8-3086f7fb7019 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:30.282422 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1470/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:47:30.987258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:30.987258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:31.001016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c68b4-cf17-4d7e-9373-fa9e61cf748f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:31.009081 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:31.010144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:31.013811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:31.013811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] PATH_INFO: `/v3/projects/f3a0a22cbf574134969550017b976494` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:31.013811 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a0a22cbf574134969550017b976494)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:31.016281 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:31.043297 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:31.173342 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a0a22cbf574134969550017b976494', 'request_id': 'req-4d23da73-ba72-424a-bb20-7af9838c1da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:31.193114 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a0a22cbf574134969550017b976494', 'request_id': 'req-4d23da73-ba72-424a-bb20-7af9838c1da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:31.294730 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a0a22cbf574134969550017b976494', 'request_id': 'req-4d23da73-ba72-424a-bb20-7af9838c1da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:31.307953 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] The token cache is being invalidate because project f3a0a22cbf574134969550017b976494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:47:31.308290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:31.308956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:47:31.310858 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1441/2911] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:47:30 2022] DELETE /identity/v3/projects/f3a0a22cbf574134969550017b976494 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:32.172707 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a271c-f619-4682-a482-7e978e11c53a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.173285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a271c-f619-4682-a482-7e978e11c53a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.173285 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a271c-f619-4682-a482-7e978e11c53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.203989 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-092a271c-f619-4682-a482-7e978e11c53a None None] MFA Rules not processed for user `ec2aa8d7d4474e5b802c91c4cb5958cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:32.205266 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-092a271c-f619-4682-a482-7e978e11c53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:32.273565 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1471/2912] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:32.278931 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:32.280389 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:32.359594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23da73-ba72-424a-bb20-7af9838c1da0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:32.364205 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:32.364821 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.365099 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.365375 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.366027 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:32.368008 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:32.467430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1442/2913] 10.222.0.16 () {60 vars in 1153 bytes} [Thu Sep 1 16:47:32 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:47:32.471454 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a271c-f619-4682-a482-7e978e11c53a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:32.484662 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a271c-f619-4682-a482-7e978e11c53a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:32.538211 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:32.547807 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:32.548604 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.548883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.552256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.552256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:32.552256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:32.671792 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1472/2914] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Sep 1 16:47:32 2022] POST /identity/v3/users => generated 295 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 01 16:47:32.682956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:32.687917 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433857de-6adb-4f69-96f3-1ba6c1744019 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:32.692633 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:32.693242 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.693516 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.693783 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.694459 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:32.696273 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:32.708426 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1443/2915] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:32.717302 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:32.728172 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942d93c-7279-4594-a04b-3068464d9563 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:32.733257 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:32.733963 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.734245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.734524 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] PATH_INFO: `/v3/projects/9f4c45b07a614aaebaea627f11eb5362/users/632d2bf4c92a414fb541a346db667865/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.752282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4c45b07a614aaebaea627f11eb5362, user_id=632d2bf4c92a414fb541a346db667865, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:32.755016 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:32.854826 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1473/2916] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:47:32 2022] PUT /identity/v3/projects/9f4c45b07a614aaebaea627f11eb5362/users/632d2bf4c92a414fb541a346db667865/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:32.859054 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:32.870411 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465516bd-e5ea-4eb3-ba18-35755720ef39 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:32.960461 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:32.961098 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:32.961380 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:32.961653 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:32.962366 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:32.964246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf686fc2-8be0-4d90-986e-fcf2dc31ce40 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:32.975868 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1444/2917] 10.222.0.16 () {58 vars in 1122 bytes} [Thu Sep 1 16:47:32 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 01 16:47:32.980201 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:33.001063 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2387a0-affa-4383-bf9d-1b6ad2ecde45 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:33.006053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:33.006828 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:33.007115 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:33.007431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] PATH_INFO: `/v3/projects/9f4c45b07a614aaebaea627f11eb5362/users/632d2bf4c92a414fb541a346db667865/roles/9730e643e75447a485d531fe8f00c6c5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:33.009726 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4c45b07a614aaebaea627f11eb5362, user_id=632d2bf4c92a414fb541a346db667865, role_id=9730e643e75447a485d531fe8f00c6c5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:33.013833 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:33.026748 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1474/2918] 10.222.0.16 () {60 vars in 1483 bytes} [Thu Sep 1 16:47:32 2022] PUT /identity/v3/projects/9f4c45b07a614aaebaea627f11eb5362/users/632d2bf4c92a414fb541a346db667865/roles/9730e643e75447a485d531fe8f00c6c5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:47:39.118870 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba31f-2bbc-4b03-b147-e14dfc380cbc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:39.120049 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba31f-2bbc-4b03-b147-e14dfc380cbc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:39.120376 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ba31f-2bbc-4b03-b147-e14dfc380cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:39.167444 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d62ba31f-2bbc-4b03-b147-e14dfc380cbc None None] MFA Rules not processed for user `632d2bf4c92a414fb541a346db667865`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:47:39.168790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d62ba31f-2bbc-4b03-b147-e14dfc380cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:39.175490 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:39.275353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1445/2919] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:47:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:47:39.289241 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:47:39.291682 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:47:39.402931 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c066ab-1579-4727-87d1-403c812de1a0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:47:39.408734 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:47:39.409437 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:47:39.409716 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:47:39.409986 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:47:39.488243 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:47:39.490015 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:47:39.499883 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1475/2920] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:47:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:47:49.458913 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=27 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:47:49.495293 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:07.135493 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:07.136065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:07.136065 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:07.157597 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] MFA Rules not processed for user `ec2aa8d7d4474e5b802c91c4cb5958cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:07.158864 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:07.166587 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1446/2921] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:48:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:08.293120 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:08.296705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:08.384146 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f864f-2337-442c-9770-03b72f851fca service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:08.390824 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:08.391917 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:08.392415 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:08.392902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:08.395758 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:08.419290 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:08.420759 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:08.433771 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1476/2922] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:09.551228 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:09.552830 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:09.566857 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464e0ce-1b7c-4d26-bc33-7411941317b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:09.571006 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:09.571668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:09.571955 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:09.572229 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:09.574780 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:09.591863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:09.593433 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633e23f-3de8-4cc3-bbdd-86b8f40bb148 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:09.607463 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1447/2923] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:18.427444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=26 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:19.600956 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:29.061025 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:29.070259 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfd9788-a39e-4d69-9dbb-22179730025d service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:29.074747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:29.075386 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:29.075661 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:29.075928 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:29.085356 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:29.087592 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255866-18c7-42fc-9e5a-73b019743976 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:29.098420 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1477/2924] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:36.974535 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3f50-95cd-443e-b04b-8e8badbaa932 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:36.975564 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3f50-95cd-443e-b04b-8e8badbaa932 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:36.975840 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3f50-95cd-443e-b04b-8e8badbaa932 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:36.977417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1448/2925] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:48:36.983668 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f55807-3d50-45de-af57-ce0738dfb7cf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:36.984190 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f55807-3d50-45de-af57-ce0738dfb7cf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:36.984311 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f55807-3d50-45de-af57-ce0738dfb7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:37.029853 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-08f55807-3d50-45de-af57-ce0738dfb7cf None None] MFA Rules not processed for user `793f922871f04a6480c571a073534d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:37.041242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-08f55807-3d50-45de-af57-ce0738dfb7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:37.041242 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1478/2926] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:48:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:37.050000 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3f50-95cd-443e-b04b-8e8badbaa932 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:37.061683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3f50-95cd-443e-b04b-8e8badbaa932 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:37.065431 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:37.066035 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:37.066311 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:37.066584 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:37.075805 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:37.077275 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfedec0-36f2-40b7-a512-0f36994c621d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:37.089064 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1449/2927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:37.177868 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cf42-cd1b-48c8-b3c8-86b36de908b1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:37.178180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cf42-cd1b-48c8-b3c8-86b36de908b1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:37.178457 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cf42-cd1b-48c8-b3c8-86b36de908b1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:37.180907 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1479/2928] 10.222.0.16 () {54 vars in 921 bytes} [Thu Sep 1 16:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 01 16:48:37.186505 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:37.186856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:37.187293 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:37.227793 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] MFA Rules not processed for user `68764b02b584408090a0c12a06b50445`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:48:37.229067 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:37.239435 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1450/2929] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Sep 1 16:48:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:48:37.255580 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cf42-cd1b-48c8-b3c8-86b36de908b1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:37.270295 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cf42-cd1b-48c8-b3c8-86b36de908b1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:37.273938 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:37.274538 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:37.274814 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:37.275107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:37.285552 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:37.286984 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:37.297901 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1480/2930] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:48:47.224719 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=15 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:47.293113 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=21 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:48:53.886888 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:48:53.896502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00b76d-2f37-4001-ba4c-34bd68294267 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:48:53.900480 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:48:53.901076 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:48:53.901349 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:48:53.901618 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:48:53.903399 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:48:53.913485 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:48:53.914964 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:48:53.923414 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1451/2931] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:48:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:49:03.919668 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:44.479745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:44.488786 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe539ca-0944-4d79-af05-4c57c3b1fd9f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:44.492641 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:44.493256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:44.493621 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:44.493966 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:44.495719 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:44.557781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:44.559264 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:44.569087 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1481/2932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:50:45.476506 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:45.478257 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:45.509993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39bb58-3b24-43a8-8808-23ab89f35e91 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:45.519863 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:45.523270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:45.523270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:45.523270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:45.524566 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:45.541382 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:45.544339 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:45.560499 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1452/2933] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:50:51.049362 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:51.059162 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83372f8-d3b1-4664-88ee-539330754fa8 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:51.062802 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:51.063619 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:51.064012 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:51.064312 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:51.066075 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:51.133281 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:51.135666 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:51.145692 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1482/2934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:50:54.542356 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:54.543837 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:54.625735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3756b8-f59d-40a6-8422-e02d421058e7 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:54.631581 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:54.632253 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:54.632543 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:54.632817 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:54.635234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:54.647696 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:54.649472 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:54.661239 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1453/2935] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:50:58.134571 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:58.148030 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49475264-1ea3-49e8-95c9-3b8abcf9cc11 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:58.154086 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:58.154705 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:58.154981 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:58.155291 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:58.157261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:58.170755 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:58.175492 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:58.183539 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1483/2936] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:50:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 01 16:50:58.260918 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:58.263016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:58.279636 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84294a22-2bb3-4653-b39b-a122bd7eee7f service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:58.284743 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:58.285509 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:58.285798 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:58.286077 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] PATH_INFO: `/v3/users/edda03844073424d89deff3eafeae7cb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:58.287270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edda03844073424d89deff3eafeae7cb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:58.289210 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:58.467393 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edda03844073424d89deff3eafeae7cb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:58.468041 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edda03844073424d89deff3eafeae7cb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:58.536651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edda03844073424d89deff3eafeae7cb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:58.544766 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edda03844073424d89deff3eafeae7cb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:58.559216 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edda03844073424d89deff3eafeae7cb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:58.565778 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1454/2937] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:50:58 2022] DELETE /identity/v3/users/edda03844073424d89deff3eafeae7cb => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:59.071633 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:59.091068 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73beb9e-4c11-43cd-9bc4-ee77b55684ba service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:59.172865 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:59.173553 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:59.173663 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:59.173934 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] PATH_INFO: `/v3/projects/a58b7358110c43ebbfbb5b6de842c94b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:59.175076 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58b7358110c43ebbfbb5b6de842c94b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:59.177038 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:59.202256 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:59.288251 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58b7358110c43ebbfbb5b6de842c94b', 'request_id': 'req-b6077d92-14e3-4c4c-a43b-3994e677911a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.299366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58b7358110c43ebbfbb5b6de842c94b', 'request_id': 'req-b6077d92-14e3-4c4c-a43b-3994e677911a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.360320 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58b7358110c43ebbfbb5b6de842c94b', 'request_id': 'req-b6077d92-14e3-4c4c-a43b-3994e677911a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.373097 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] The token cache is being invalidate because project a58b7358110c43ebbfbb5b6de842c94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:50:59.373502 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.374519 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.376336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1484/2938] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:50:59 2022] DELETE /identity/v3/projects/a58b7358110c43ebbfbb5b6de842c94b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:59.380991 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:59.382789 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:50:59.397925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada23a7c-8279-4656-b0f2-2b7a5bc8aba4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:50:59.448683 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:50:59.463896 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:50:59.464540 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:50:59.464818 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:50:59.465086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] PATH_INFO: `/v3/users/dfe61da74ec9481683f5aca80ad514a3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:50:59.466259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe61da74ec9481683f5aca80ad514a3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:50:59.468241 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:50:59.598501 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe61da74ec9481683f5aca80ad514a3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.599484 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe61da74ec9481683f5aca80ad514a3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.674546 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe61da74ec9481683f5aca80ad514a3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.684191 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe61da74ec9481683f5aca80ad514a3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.697119 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0b104d77-a32b-4bad-9604-31932f731df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe61da74ec9481683f5aca80ad514a3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:50:59.703523 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1455/2939] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:50:59 2022] DELETE /identity/v3/users/dfe61da74ec9481683f5aca80ad514a3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:50:59.990918 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:50:59.994081 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:00.010051 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6077d92-14e3-4c4c-a43b-3994e677911a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:00.083000 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:00.083691 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:00.083973 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:00.084245 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] PATH_INFO: `/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:00.085408 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a3b8290b004e9ab2ecb1bce2b47388)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:00.088434 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:00.196846 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a3b8290b004e9ab2ecb1bce2b47388', 'request_id': 'req-4530afc1-10aa-4e15-8501-abc06882f7b7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:00.204400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a3b8290b004e9ab2ecb1bce2b47388', 'request_id': 'req-4530afc1-10aa-4e15-8501-abc06882f7b7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:00.285237 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a3b8290b004e9ab2ecb1bce2b47388', 'request_id': 'req-4530afc1-10aa-4e15-8501-abc06882f7b7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:00.298560 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] The token cache is being invalidate because project f5a3b8290b004e9ab2ecb1bce2b47388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:51:00.298872 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:00.299531 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:51:00.301052 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1485/2940] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:50:59 2022] DELETE /identity/v3/projects/f5a3b8290b004e9ab2ecb1bce2b47388 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:51:09.699594 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:10.294400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:14.918258 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:14.918781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:14.918781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:14.949769 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] MFA Rules not processed for user `ec2aa8d7d4474e5b802c91c4cb5958cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:51:14.950984 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:15.024465 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1456/2941] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:51:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:51:15.359776 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:15.361261 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:15.442808 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530afc1-10aa-4e15-8501-abc06882f7b7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:15.446409 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:15.447609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:15.447871 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:15.448182 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:15.450743 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:15.461637 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:15.463932 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:15.473280 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1486/2942] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:16.216050 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:16.217645 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:16.311495 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8a601-db40-4a35-a5dc-4b4621214f09 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:16.316363 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:16.316972 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:16.317249 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:16.317517 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:16.319270 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:16.332260 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:16.333851 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:16.345417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1457/2943] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:25.468964 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=35 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:26.340400 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=72 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:32.456382 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:32.458517 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:32.585595 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9d1ad-d209-4ca3-b30a-55b0ae014b30 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:32.591858 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:32.592461 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:32.592735 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:32.593006 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:32.594728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:32.609831 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:32.611902 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:32.624609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1487/2944] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:38.523290 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:51:38.537804 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc722b3-9c1f-4d4b-a7ab-8d2a614efaaa service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:51:38.544364 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:51:38.544987 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:51:38.545264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:51:38.545534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:51:38.547487 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:51:38.560604 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:51:38.562215 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:51:38.574230 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1458/2945] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:51:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:51:42.618496 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=46 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:51:48.569600 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:52:07.210923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:52:07.232781 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce465908-9f74-4f47-a5e5-c6ea8018cf1c service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:52:07.238180 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:52:07.238774 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:52:07.239049 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:52:07.239342 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:52:07.241069 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:52:07.267983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:52:07.269899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756126a6-8bd0-4c1e-b3d6-9f3f45c55610 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:52:07.279983 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1488/2946] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:52:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:52:17.275594 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=15 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:41.047430 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:41.048877 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:41.131952 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb367165-5e8b-4761-b54e-216f0f0e8d4d service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:41.135615 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:41.136226 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:41.136502 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:41.136771 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:41.138922 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:41.151124 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:41.152607 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:41.162788 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1459/2947] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:51.157735 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=35 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:53:56.043629 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:56.044223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:56.044223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:56.063968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] MFA Rules not processed for user `ec2aa8d7d4474e5b802c91c4cb5958cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 01 16:53:56.065178 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:56.072500 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1489/2948] 10.222.0.16 () {58 vars in 958 bytes} [Thu Sep 1 16:53:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 01 16:53:56.130439 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:56.131923 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:56.145246 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cfcd1-de08-466a-9669-3068ea9c2364 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:56.149534 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a081c08049447949477e73e0891f1e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:56.150166 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:56.150419 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:56.150688 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:56.154958 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:56.169326 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:56.170781 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:56.187694 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1460/2949] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:56.485394 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:56.486818 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:56.527392 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e763722-f234-4b21-8b56-6dcdd92acf57 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:56.534488 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:56.535123 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:56.535444 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:56.536375 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:56.538602 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:56.549763 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:56.551282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:56.560699 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1490/2950] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:53:57.397113 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:53:57.406728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2369ac5-f1c5-4e1c-ab16-7ee8e7a7dfc9 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:53:57.410656 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:53:57.411559 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:53:57.411838 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:53:57.412108 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:53:57.413925 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:53:57.428635 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:53:57.430086 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:53:57.442252 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1461/2951] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:53:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:54:00.594103 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:54:00.612618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b9398-9b07-4935-8963-5a1ba1c58ebe service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:54:00.612618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:54:00.612618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:54:00.612618 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:54:00.613107 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:54:00.615131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:54:00.626163 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:54:00.627655 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:54:00.637540 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1491/2952] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:54:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:54:07.436671 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:54:10.633234 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=24 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:54:17.636916 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:54:17.646089 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c44b0-03e9-4eac-b7eb-74f359e81586 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:54:17.649861 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:54:17.650475 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:54:17.650751 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:54:17.651016 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:54:17.652824 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:54:17.721062 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:54:17.722525 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:54:17.732548 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1462/2953] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:54:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:54:27.728073 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:24.001503 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:24.015441 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8afe46-b07c-4dab-a16a-9504548a99cb service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:24.021033 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68764b02b584408090a0c12a06b50445', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:24.023631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:24.023631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:24.023631 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:24.023997 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:24.036223 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:24.037821 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:24.049470 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1492/2954] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:34.043799 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:37.032623 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:37.041447 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f08b-05f2-4f47-863c-d79753f5198a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:37.045234 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793f922871f04a6480c571a073534d3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:37.045856 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:37.046149 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:37.046406 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:37.048776 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:37.136304 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:37.137765 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:37.158421 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1463/2955] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:40.462013 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:40.473884 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad0c79-51ab-4cad-8ef9-e490ca325ce5 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:40.479728 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:40.480366 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:40.480650 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:40.480923 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:40.482744 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:40.495131 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:40.496739 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:40.508708 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1493/2956] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:56:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:56:44.192654 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:44.195193 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:44.207728 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c74fae-45e8-4972-9f45-3ce55a09b341 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:44.212826 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:44.213462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:44.213744 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:44.214019 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] PATH_INFO: `/v3/users/ec2aa8d7d4474e5b802c91c4cb5958cd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:44.215144 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2aa8d7d4474e5b802c91c4cb5958cd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:44.217105 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:44.407842 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2aa8d7d4474e5b802c91c4cb5958cd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:44.408537 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2aa8d7d4474e5b802c91c4cb5958cd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:44.495310 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2aa8d7d4474e5b802c91c4cb5958cd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:44.503422 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2aa8d7d4474e5b802c91c4cb5958cd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:44.515117 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2aa8d7d4474e5b802c91c4cb5958cd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:44.521126 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1464/2957] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:56:44 2022] DELETE /identity/v3/users/ec2aa8d7d4474e5b802c91c4cb5958cd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:44.788442 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:44.800060 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d40724-1fc0-4e1c-968b-cff2b3e8cfa8 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:44.859417 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:44.860165 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:44.860991 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:44.861418 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] PATH_INFO: `/v3/projects/38f1672c9b5147fe8fef8659e7953fef` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:44.862564 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38f1672c9b5147fe8fef8659e7953fef)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:44.864544 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:45.010053 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38f1672c9b5147fe8fef8659e7953fef', 'request_id': 'req-7fc93577-a12b-4c09-a8cb-ca6814f6e200'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.016526 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38f1672c9b5147fe8fef8659e7953fef', 'request_id': 'req-7fc93577-a12b-4c09-a8cb-ca6814f6e200'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.089514 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38f1672c9b5147fe8fef8659e7953fef', 'request_id': 'req-7fc93577-a12b-4c09-a8cb-ca6814f6e200'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.100910 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] The token cache is being invalidate because project 38f1672c9b5147fe8fef8659e7953fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:56:45.101197 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.101899 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.103431 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1494/2958] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:56:44 2022] DELETE /identity/v3/projects/38f1672c9b5147fe8fef8659e7953fef => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:45.107476 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:45.114663 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:45.122571 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74714a-f208-48b4-9d22-2f77c69adc52 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:45.156462 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:45.180496 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:45.181134 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:45.181417 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:45.181702 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] PATH_INFO: `/v3/users/632d2bf4c92a414fb541a346db667865` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:45.182822 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=632d2bf4c92a414fb541a346db667865)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:45.184790 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:45.400993 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '632d2bf4c92a414fb541a346db667865'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.401651 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '632d2bf4c92a414fb541a346db667865'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.472224 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '632d2bf4c92a414fb541a346db667865'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.479553 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '632d2bf4c92a414fb541a346db667865'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.491353 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '632d2bf4c92a414fb541a346db667865'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.497201 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1465/2959] 10.222.0.16 () {58 vars in 1224 bytes} [Thu Sep 1 16:56:45 2022] DELETE /identity/v3/users/632d2bf4c92a414fb541a346db667865 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:45.799857 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:56:45.801313 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:56:45.815491 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc93577-a12b-4c09-a8cb-ca6814f6e200 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:56:45.872400 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9fb5099a6c24abebce19b1a9dfe02b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18a3d39073f24e449904bb491fbe8869', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:56:45.873031 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:56:45.873349 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:56:45.873635 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] PATH_INFO: `/v3/projects/9f4c45b07a614aaebaea627f11eb5362` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:56:45.874745 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4c45b07a614aaebaea627f11eb5362)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:56:45.877427 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:56:45.898336 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:45.968992 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4c45b07a614aaebaea627f11eb5362', 'request_id': 'req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:45.975322 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4c45b07a614aaebaea627f11eb5362', 'request_id': 'req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:46.066282 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4c45b07a614aaebaea627f11eb5362', 'request_id': 'req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:46.077476 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] The token cache is being invalidate because project 9f4c45b07a614aaebaea627f11eb5362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 01 16:56:46.077747 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:46.078368 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 01 16:56:46.080176 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1495/2960] 10.222.0.16 () {58 vars in 1233 bytes} [Thu Sep 1 16:56:45 2022] DELETE /identity/v3/projects/9f4c45b07a614aaebaea627f11eb5362 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 01 16:56:55.493787 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:56:56.101067 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 16:57:13.232844 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 16:57:13.234264 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:57:13.305029 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee87c6-c5dd-4736-a5c3-e2acd35c7f5e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 16:57:13.363662 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 16:57:13.364316 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 16:57:13.364591 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 16:57:13.364859 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 16:57:13.366707 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 16:57:13.378640 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 16:57:13.380259 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99babadc-76bb-454c-8033-adbf80ce918b service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 16:57:13.390143 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1466/2961] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 16:57:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 16:57:23.385873 dv-cinder-836027-9-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=58 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 01 17:02:15.242184 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 01 17:02:15.243721 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 01 17:02:15.330968 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b5e39-9d04-40bb-bb30-0d0ff6d727be admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 01 17:02:15.336098 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0688b8d8c0aa46818633fbc17dfb58d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7b628bcf00bc4a80be6a5ef7a15084c3', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 01 17:02:15.336701 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 01 17:02:15.336980 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 01 17:02:15.337271 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 01 17:02:15.345998 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 01 17:02:15.347609 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38695f9a-fad7-471e-b9dd-c58ada7a48f0 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 01 17:02:15.357390 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1496/2962] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Sep 1 17:02:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 01 17:02:25.353128 dv-cinder-836027-9-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=37 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}